2024-05-03T17:36:41.549 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-03T17:36:41.549 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} --archive /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410 --owner scheduled_rishabh@teuthology --verbose --name rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi -- /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410/orig.config.yaml 2024-05-03T17:36:41.598 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-03T17:36:41.692 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410 branch: wip-rishabh-testing-20240501.193033 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} email: null first_in_suite: false job_id: '7687410' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240501.193033 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240501.193033 sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1698 sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea sleep_before_teardown: 0 subset: 434/512 suite: fs suite_branch: wip-rishabh-testing-20240501.193033 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGs35vtwL+0yN/J032ku2Vt6SdDjMgN8+RiuzTk3cZUfyPOCWi8tOZ5LJUUmfidtytXDavqgYJU1bHin+WcvHfo= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOH20P6Jxshe2lqJN0Pt/+RuGGcCp2X+1Fns/lvH7xoS6GQFxJP8Scvtex/LQ/7+BiwnIpZuoNUHPz4VCPgrWzo= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDTAhAyIIG5W3tltDH9m3ykiOIeWdXDXdlSrM1UbDQNYe1xtmP+UND6uYbxt3buulING/ZqQTsAFAsJP4FkjwJU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-03_13:56:19 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-03T17:36:41.692 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa; will attempt to use it 2024-05-03T17:36:41.693 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa/tasks 2024-05-03T17:36:41.693 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-03T17:36:41.695 INFO:teuthology.task.internal:Checking packages... 2024-05-03T17:36:41.716 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9a047fe7599a549042d06a3fb5b6813a203eabea' 2024-05-03T17:36:41.716 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-03T17:36:41.716 INFO:teuthology.packaging:ref: None 2024-05-03T17:36:41.717 INFO:teuthology.packaging:tag: None 2024-05-03T17:36:41.717 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240501.193033 2024-05-03T17:36:41.717 INFO:teuthology.packaging:sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:36:41.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240501.193033 2024-05-03T17:36:41.948 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3450.g9a047fe7 2024-05-03T17:36:41.949 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-03T17:36:41.956 INFO:teuthology.task.internal:no buildpackages task found 2024-05-03T17:36:41.956 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-03T17:36:41.962 INFO:teuthology.task.internal:Saving configuration 2024-05-03T17:36:41.983 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-03T17:36:41.989 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T17:36:42.015 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 17:31:24.231015', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGs35vtwL+0yN/J032ku2Vt6SdDjMgN8+RiuzTk3cZUfyPOCWi8tOZ5LJUUmfidtytXDavqgYJU1bHin+WcvHfo='} 2024-05-03T17:36:42.034 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 17:31:24.227649', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOH20P6Jxshe2lqJN0Pt/+RuGGcCp2X+1Fns/lvH7xoS6GQFxJP8Scvtex/LQ/7+BiwnIpZuoNUHPz4VCPgrWzo='} 2024-05-03T17:36:42.052 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 17:31:24.229542', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDTAhAyIIG5W3tltDH9m3ykiOIeWdXDXdlSrM1UbDQNYe1xtmP+UND6uYbxt3buulING/ZqQTsAFAsJP4FkjwJU='} 2024-05-03T17:36:42.052 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-03T17:36:42.057 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-03T17:36:42.057 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-03T17:36:42.058 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-03T17:36:42.058 INFO:teuthology.run_tasks:Running task console_log... 2024-05-03T17:36:42.150 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f924857f160>, signals=[15]) 2024-05-03T17:36:42.151 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-03T17:36:42.157 INFO:teuthology.task.internal:Opening connections... 2024-05-03T17:36:42.157 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-05-03T17:36:42.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T17:36:42.238 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-05-03T17:36:42.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T17:36:42.311 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-05-03T17:36:42.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T17:36:42.386 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-03T17:36:42.393 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-05-03T17:36:42.412 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-05-03T17:36:42.412 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-05-03T17:36:42.468 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-05-03T17:36:42.468 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T17:36:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T17:36:42.470 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-05-03T17:36:42.489 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-05-03T17:36:42.506 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-05-03T17:36:42.507 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-05-03T17:36:42.563 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-05-03T17:36:42.563 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-05-03T17:36:42.563 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-05-03T17:36:42.563 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-05-03T17:36:42.563 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-05-03T17:36:42.563 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-05-03T17:36:42.564 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T17:36:42.564 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-05-03T17:36:42.564 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-05-03T17:36:42.564 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T17:36:42.564 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-05-03T17:36:42.564 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T17:36:42.564 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T17:36:42.564 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T17:36:42.564 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-05-03T17:36:42.589 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-05-03T17:36:42.607 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-05-03T17:36:42.607 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-05-03T17:36:42.665 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T17:36:42.666 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T17:36:42.666 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T17:36:42.666 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-05-03T17:36:42.689 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-03T17:36:42.699 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-03T17:36:42.709 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-03T17:36:42.710 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-05-03T17:36:42.712 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-05-03T17:36:42.715 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-05-03T17:36:42.731 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-03T17:36:42.737 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-03T17:36:42.737 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-05-03T17:36:42.769 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-05-03T17:36:42.771 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-05-03T17:36:42.871 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-03T17:36:42.948 INFO:teuthology.run_tasks:Running task kernel... 2024-05-03T17:36:42.963 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-05-03T17:36:42.963 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-03T17:36:42.963 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-03T17:36:42.964 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T17:36:42.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T17:36:42.981 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-05-03T17:36:43.037 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-437.el9.x86_64 2024-05-03T17:36:43.038 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.14.0-437.el9.x86_64 2024-05-03T17:36:43.038 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum install -y kernel 2024-05-03T17:36:46.378 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-05-03T17:36:49.579 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 19 MB 00:02 2024-05-03T17:36:55.013 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-03T17:36:56.581 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-05-03T17:36:57.733 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-03T17:37:02.891 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-05-03T17:37:04.789 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-03T17:37:04.789 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-03T17:37:04.884 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-05-03T17:37:04.885 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-05-03T17:37:04.885 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-05-03T17:37:05.006 DEBUG:teuthology.orchestra.run.smithi122:> echo no | sudo yum reinstall kernel || true 2024-05-03T17:37:05.507 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:03 ago on Fri 03 May 2024 05:37:02 PM UTC. 2024-05-03T17:37:05.630 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-05-03T17:37:05.643 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-05-03T17:37:05.643 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-05-03T17:37:05.644 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-05-03T17:37:05.644 INFO:teuthology.orchestra.run.smithi122.stdout:Reinstalling: 2024-05-03T17:37:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-03T17:37:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:37:05.644 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-05-03T17:37:05.644 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-05-03T17:37:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:37:05.644 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 255 k 2024-05-03T17:37:05.644 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 0 2024-05-03T17:37:05.644 INFO:teuthology.orchestra.run.smithi122.stderr:Operation aborted. 2024-05-03T17:37:05.707 INFO:teuthology.orchestra.run.smithi122.stdout:Is this ok [y/N]: 2024-05-03T17:37:05.707 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum reinstall -y kernel || true 2024-05-03T17:37:06.201 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:04 ago on Fri 03 May 2024 05:37:02 PM UTC. 2024-05-03T17:37:06.316 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-05-03T17:37:06.317 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-05-03T17:37:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-05-03T17:37:06.317 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-05-03T17:37:06.317 INFO:teuthology.orchestra.run.smithi122.stdout:Reinstalling: 2024-05-03T17:37:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-03T17:37:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:37:06.318 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-05-03T17:37:06.318 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-05-03T17:37:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:37:06.318 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 255 k 2024-05-03T17:37:06.318 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 0 2024-05-03T17:37:06.318 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-05-03T17:37:06.543 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-05-03T17:37:06.543 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-05-03T17:37:06.543 INFO:teuthology.orchestra.run.smithi122.stdout:Total 1.1 MB/s | 255 kB 00:00 2024-05-03T17:37:06.559 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-05-03T17:37:06.738 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-05-03T17:37:06.738 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-05-03T17:37:06.743 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-05-03T17:37:06.744 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-05-03T17:37:06.866 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-05-03T17:37:06.944 INFO:teuthology.orchestra.run.smithi122.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-03T17:37:06.945 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-03T17:37:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-05-03T17:37:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-03T17:37:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-03T17:37:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:37:08.385 INFO:teuthology.orchestra.run.smithi122.stdout:Reinstalled: 2024-05-03T17:37:08.385 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-5.14.0-437.el9.x86_64 2024-05-03T17:37:08.386 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:37:08.386 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-05-03T17:37:08.547 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-05-03T17:37:08.593 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-437.el9.x86_64 2024-05-03T17:37:08.593 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-05-03T17:37:08.593 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-03T17:37:08.594 DEBUG:teuthology.parallel:result is None 2024-05-03T17:37:08.594 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-03T17:37:08.603 INFO:teuthology.task.internal:Creating test directory... 2024-05-03T17:37:08.603 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T17:37:08.605 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T17:37:08.608 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T17:37:08.625 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-03T17:37:08.632 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-03T17:37:08.640 INFO:teuthology.task.internal:Creating archive directory... 2024-05-03T17:37:08.640 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T17:37:08.660 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T17:37:08.665 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T17:37:08.749 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-03T17:37:08.756 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-03T17:37:08.756 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T17:37:08.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T17:37:08.773 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T17:37:08.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T17:37:08.789 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T17:37:08.791 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T17:37:08.816 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T17:37:08.831 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T17:37:08.847 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T17:37:08.857 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T17:37:08.865 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T17:37:08.879 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T17:37:08.893 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T17:37:08.895 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-03T17:37:08.901 INFO:teuthology.task.internal:Configuring sudo... 2024-05-03T17:37:08.901 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T17:37:08.904 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T17:37:08.906 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T17:37:08.947 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-03T17:37:08.956 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-03T17:37:08.956 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T17:37:08.968 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T17:37:08.972 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T17:37:09.003 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T17:37:09.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T17:37:09.115 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T17:37:09.193 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T17:37:09.235 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:37:09.235 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T17:37:09.298 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T17:37:09.343 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T17:37:09.392 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T17:37:09.473 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T17:37:09.517 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T17:37:09.517 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T17:37:09.582 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T17:37:09.621 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T17:37:09.681 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T17:37:09.765 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T17:37:09.805 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-03T17:37:09.806 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T17:37:09.871 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-05-03T17:37:09.873 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-05-03T17:37:09.876 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-05-03T17:37:09.902 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T17:37:09.914 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T17:37:09.949 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T17:37:10.400 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-03T17:37:10.408 INFO:teuthology.task.internal:Starting timer... 2024-05-03T17:37:10.408 INFO:teuthology.run_tasks:Running task pcp... 2024-05-03T17:37:10.418 INFO:teuthology.run_tasks:Running task selinux... 2024-05-03T17:37:10.428 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-03T17:37:10.429 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-05-03T17:37:10.478 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-05-03T17:37:10.479 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-05-03T17:37:10.522 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-05-03T17:37:10.523 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-05-03T17:37:10.581 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-05-03T17:37:10.583 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-03T17:37:10.583 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-05-03T17:37:10.603 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-05-03T17:37:10.604 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-05-03T17:37:10.625 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-05-03T17:37:10.625 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-05-03T17:37:10.651 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-05-03T17:37:10.651 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi122.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-05-03T17:37:10.652 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T17:37:10.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T17:37:10.676 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T17:37:10.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T17:37:10.702 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T17:37:10.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T17:37:10.729 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-03T17:37:10.729 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-05-03T17:37:10.764 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-05-03T17:37:10.794 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-05-03T17:37:10.826 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-03T17:37:10.836 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-03T17:37:10.842 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-03T17:37:10.974 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-03T17:37:10.991 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-03T17:37:10.992 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi122.front.sepia.ceph.com,smithi149.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-03T17:41:18.461 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi122.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-05-03T17:41:18.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-05-03T17:41:18.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T17:41:18.547 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-05-03T17:41:18.625 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-05-03T17:41:18.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-03T17:41:18.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T17:41:18.702 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-05-03T17:41:18.786 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-03T17:41:18.787 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-03T17:41:18.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T17:41:18.862 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-05-03T17:41:18.940 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-03T17:41:18.940 INFO:teuthology.run_tasks:Running task clock... 2024-05-03T17:41:18.952 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-03T17:41:18.952 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T17:41:18.953 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T17:41:18.955 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T17:41:18.955 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T17:41:18.958 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T17:41:18.958 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T17:41:18.980 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T17:41:18.986 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T17:41:18.993 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T17:41:19.000 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T17:41:19.007 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T17:41:19.015 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-05-03T17:41:19.020 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T17:41:19.024 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-05-03T17:41:19.025 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-05-03T17:41:19.034 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-05-03T17:41:19.037 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T17:41:19.042 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-05-03T17:41:19.048 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T17:41:19.050 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T17:41:19.053 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-05-03T17:41:19.062 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T17:41:19.065 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T17:41:19.078 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T17:41:19.105 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-05-03T17:41:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T17:41:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-05-03T17:41:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.109 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.114 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-05-03T17:41:19.119 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T17:41:19.119 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-05-03T17:41:19.119 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.119 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.119 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.119 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.133 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-03T17:41:19.136 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T17:41:19.136 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-03T17:41:19.136 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.136 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.136 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.136 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T17:41:19.137 INFO:teuthology.run_tasks:Running task install... 2024-05-03T17:41:19.149 DEBUG:teuthology.task.install:project ceph 2024-05-03T17:41:19.149 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea'}} 2024-05-03T17:41:19.149 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea'} 2024-05-03T17:41:19.149 INFO:teuthology.task.install:Using flavor: default 2024-05-03T17:41:19.157 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-03T17:41:19.157 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-03T17:41:19.157 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T17:41:19.158 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:41:19.160 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T17:41:19.160 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:41:19.161 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T17:41:19.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:41:19.327 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T17:41:19.328 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T17:41:19.384 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T17:41:19.385 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T17:41:19.389 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T17:41:19.389 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T17:41:19.504 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T17:41:19.505 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T17:41:19.505 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T17:41:19.531 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T17:41:19.531 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T17:41:19.555 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T17:41:19.555 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:41:19.555 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T17:41:19.560 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T17:41:19.560 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-03T17:41:19.560 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T17:41:19.580 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T17:41:19.580 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T17:41:19.586 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T17:41:19.586 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T17:41:19.597 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T17:41:19.644 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T17:41:19.652 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T17:41:19.670 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T17:41:19.717 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T17:41:19.726 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T17:41:19.734 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-05-03T17:41:19.735 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-05-03T17:41:19.779 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-05-03T17:41:19.781 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-05-03T17:41:19.789 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-05-03T17:41:19.791 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-05-03T17:41:19.959 INFO:teuthology.orchestra.run.smithi149.stdout:50 files removed 2024-05-03T17:41:19.986 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T17:41:20.005 INFO:teuthology.orchestra.run.smithi122.stdout:50 files removed 2024-05-03T17:41:20.016 INFO:teuthology.orchestra.run.smithi192.stdout:50 files removed 2024-05-03T17:41:20.033 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T17:41:20.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T17:41:20.653 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 241 kB/s | 84 kB 00:00 2024-05-03T17:41:20.889 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 131 kB/s | 84 kB 00:00 2024-05-03T17:41:20.978 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 123 kB/s | 84 kB 00:00 2024-05-03T17:41:21.187 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-05-03T17:41:21.248 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-05-03T17:41:21.362 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-05-03T17:41:21.579 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-05-03T17:41:21.729 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-05-03T17:41:21.798 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2024-05-03T17:41:22.188 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-05-03T17:41:22.549 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-05-03T17:41:22.773 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.0 MB 00:00 2024-05-03T17:41:24.156 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-05-03T17:41:26.948 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 19 MB 00:03 2024-05-03T17:41:27.524 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 19 MB 00:03 2024-05-03T17:41:28.631 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.2 MB 00:00 2024-05-03T17:41:30.349 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-05-03T17:41:31.266 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.2 MB 00:00 2024-05-03T17:41:31.625 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-03T17:41:32.733 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-05-03T17:41:33.201 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-05-03T17:41:33.794 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-05-03T17:41:37.343 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 3.1 MB/s | 21 MB 00:06 2024-05-03T17:41:38.819 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-03T17:41:40.953 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T17:41:40.954 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T17:41:40.965 INFO:teuthology.orchestra.run.smithi122.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T17:41:40.966 INFO:teuthology.orchestra.run.smithi122.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T17:41:40.967 INFO:teuthology.orchestra.run.smithi122.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T17:41:40.968 INFO:teuthology.orchestra.run.smithi122.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T17:41:40.969 INFO:teuthology.orchestra.run.smithi122.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T17:41:40.971 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T17:41:40.972 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T17:41:41.033 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-05-03T17:41:41.042 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-05-03T17:41:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-05-03T17:41:41.042 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-05-03T17:41:41.042 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-05-03T17:41:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T17:41:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T17:41:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T17:41:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T17:41:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T17:41:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T17:41:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T17:41:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T17:41:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T17:41:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T17:41:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T17:41:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T17:41:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T17:41:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T17:41:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T17:41:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T17:41:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-05-03T17:41:41.053 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-05-03T17:41:41.054 INFO:teuthology.orchestra.run.smithi122.stdout:Install 148 Packages 2024-05-03T17:41:41.054 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 4 Packages 2024-05-03T17:41:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:41:41.054 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 222 M 2024-05-03T17:41:41.054 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-05-03T17:41:41.862 INFO:teuthology.orchestra.run.smithi122.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-05-03T17:41:42.179 INFO:teuthology.orchestra.run.smithi122.stdout:(2/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 2.7 MB/s | 891 kB 00:00 2024-05-03T17:41:42.304 INFO:teuthology.orchestra.run.smithi122.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 1.1 MB/s | 145 kB 00:00 2024-05-03T17:41:42.408 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-03T17:41:42.621 INFO:teuthology.orchestra.run.smithi122.stdout:(4/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 7.5 MB/s | 2.4 MB 00:00 2024-05-03T17:41:42.688 INFO:teuthology.orchestra.run.smithi122.stdout:(5/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 5.2 MB/s | 5.4 MB 00:01 2024-05-03T17:41:42.855 INFO:teuthology.orchestra.run.smithi122.stdout:(6/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 6.5 MB/s | 1.5 MB 00:00 2024-05-03T17:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout:(7/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 7.7 MB/s | 4.7 MB 00:00 2024-05-03T17:41:43.789 INFO:teuthology.orchestra.run.smithi122.stdout:(8/152): ceph-common-19.0.0-3450.g9a047fe7.el9. 9.9 MB/s | 21 MB 00:02 2024-05-03T17:41:43.873 INFO:teuthology.orchestra.run.smithi122.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 302 kB/s | 25 kB 00:00 2024-05-03T17:41:44.457 INFO:teuthology.orchestra.run.smithi122.stdout:(10/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el 9.1 MB/s | 11 MB 00:01 2024-05-03T17:41:44.551 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T17:41:44.552 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T17:41:44.557 INFO:teuthology.orchestra.run.smithi122.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 2.2 MB/s | 225 kB 00:00 2024-05-03T17:41:44.563 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T17:41:44.563 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T17:41:44.564 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T17:41:44.565 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T17:41:44.567 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T17:41:44.569 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T17:41:44.570 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T17:41:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-03T17:41:44.649 INFO:teuthology.orchestra.run.smithi122.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 295 kB/s | 27 kB 00:00 2024-05-03T17:41:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-03T17:41:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T17:41:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T17:41:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T17:41:44.656 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T17:41:44.657 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T17:41:44.658 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T17:41:44.659 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T17:41:44.660 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T17:41:44.661 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T17:41:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T17:41:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T17:41:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T17:41:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T17:41:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T17:41:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T17:41:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T17:41:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T17:41:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T17:41:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T17:41:44.662 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T17:41:44.663 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T17:41:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T17:41:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T17:41:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T17:41:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T17:41:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-05-03T17:41:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T17:41:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:41:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-03T17:41:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-03T17:41:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:Install 148 Packages 2024-05-03T17:41:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 4 Packages 2024-05-03T17:41:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:41:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 222 M 2024-05-03T17:41:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-03T17:41:44.824 INFO:teuthology.orchestra.run.smithi122.stdout:(13/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x8 8.5 MB/s | 17 MB 00:01 2024-05-03T17:41:44.858 INFO:teuthology.orchestra.run.smithi122.stdout:(14/152): libcephfs-devel-19.0.0-3450.g9a047fe7 152 kB/s | 32 kB 00:00 2024-05-03T17:41:44.916 INFO:teuthology.orchestra.run.smithi122.stdout:(15/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 7.9 MB/s | 747 kB 00:00 2024-05-03T17:41:44.942 INFO:teuthology.orchestra.run.smithi122.stdout:(16/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 1.9 MB/s | 163 kB 00:00 2024-05-03T17:41:44.992 INFO:teuthology.orchestra.run.smithi122.stdout:(17/152): librados-devel-19.0.0-3450.g9a047fe7. 1.6 MB/s | 127 kB 00:00 2024-05-03T17:41:45.026 INFO:teuthology.orchestra.run.smithi122.stdout:(18/152): libradosstriper1-19.0.0-3450.g9a047fe 5.8 MB/s | 500 kB 00:00 2024-05-03T17:41:45.118 INFO:teuthology.orchestra.run.smithi122.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 492 kB/s | 45 kB 00:00 2024-05-03T17:41:45.218 INFO:teuthology.orchestra.run.smithi122.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 1.3 MB/s | 135 kB 00:00 2024-05-03T17:41:45.318 INFO:teuthology.orchestra.run.smithi122.stdout:(21/152): python3-cephfs-19.0.0-3450.g9a047fe7. 1.6 MB/s | 161 kB 00:00 2024-05-03T17:41:45.427 INFO:teuthology.orchestra.run.smithi122.stdout:(22/152): python3-rados-19.0.0-3450.g9a047fe7.e 2.9 MB/s | 320 kB 00:00 2024-05-03T17:41:45.551 INFO:teuthology.orchestra.run.smithi192.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-03T17:41:45.569 INFO:teuthology.orchestra.run.smithi122.stdout:(23/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 9.3 MB/s | 5.4 MB 00:00 2024-05-03T17:41:45.594 INFO:teuthology.orchestra.run.smithi122.stdout:(24/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 1.7 MB/s | 298 kB 00:00 2024-05-03T17:41:45.653 INFO:teuthology.orchestra.run.smithi122.stdout:(25/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 1.2 MB/s | 100 kB 00:00 2024-05-03T17:41:45.678 INFO:teuthology.orchestra.run.smithi122.stdout:(26/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-03T17:41:45.804 INFO:teuthology.orchestra.run.smithi122.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 1.3 MB/s | 170 kB 00:00 2024-05-03T17:41:45.921 INFO:teuthology.orchestra.run.smithi122.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3450.g 293 kB/s | 34 kB 00:00 2024-05-03T17:41:46.030 INFO:teuthology.orchestra.run.smithi122.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 1.3 MB/s | 142 kB 00:00 2024-05-03T17:41:46.130 INFO:teuthology.orchestra.run.smithi122.stdout:(30/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 6.5 MB/s | 3.1 MB 00:00 2024-05-03T17:41:46.577 INFO:teuthology.orchestra.run.smithi192.stdout:(2/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 869 kB/s | 891 kB 00:01 2024-05-03T17:41:46.614 INFO:teuthology.orchestra.run.smithi122.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 6.1 MB/s | 3.5 MB 00:00 2024-05-03T17:41:46.714 INFO:teuthology.orchestra.run.smithi122.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 2.7 MB/s | 274 kB 00:00 2024-05-03T17:41:46.718 INFO:teuthology.orchestra.run.smithi192.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 1.0 MB/s | 145 kB 00:00 2024-05-03T17:41:46.806 INFO:teuthology.orchestra.run.smithi122.stdout:(33/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 538 kB/s | 49 kB 00:00 2024-05-03T17:41:46.889 INFO:teuthology.orchestra.run.smithi122.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3450.g9 198 kB/s | 16 kB 00:00 2024-05-03T17:41:47.007 INFO:teuthology.orchestra.run.smithi122.stdout:(35/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 2.3 MB/s | 273 kB 00:00 2024-05-03T17:41:47.085 INFO:teuthology.orchestra.run.smithi192.stdout:(4/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 2.8 MB/s | 5.4 MB 00:01 2024-05-03T17:41:47.182 INFO:teuthology.orchestra.run.smithi122.stdout:(36/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 4.3 MB/s | 763 kB 00:00 2024-05-03T17:41:47.266 INFO:teuthology.orchestra.run.smithi122.stdout:(37/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 219 kB/s | 18 kB 00:00 2024-05-03T17:41:47.602 INFO:teuthology.orchestra.run.smithi192.stdout:(5/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-05-03T17:41:47.644 INFO:teuthology.orchestra.run.smithi192.stdout:(6/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 2.6 MB/s | 2.4 MB 00:00 2024-05-03T17:41:47.758 INFO:teuthology.orchestra.run.smithi122.stdout:(38/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 13 MB/s | 49 MB 00:03 2024-05-03T17:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 4.3 MB/s | 7.4 MB 00:01 2024-05-03T17:41:48.392 INFO:teuthology.orchestra.run.smithi122.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 71 kB/s | 80 kB 00:01 2024-05-03T17:41:48.626 INFO:teuthology.orchestra.run.smithi122.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 252 kB/s | 195 kB 00:00 2024-05-03T17:41:48.669 INFO:teuthology.orchestra.run.smithi192.stdout:(7/152): ceph-common-19.0.0-3450.g9a047fe7.el9. 6.1 MB/s | 21 MB 00:03 2024-05-03T17:41:48.701 INFO:teuthology.orchestra.run.smithi122.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 852 kB/s | 803 kB 00:00 2024-05-03T17:41:48.736 INFO:teuthology.orchestra.run.smithi192.stdout:(8/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-05-03T17:41:48.803 INFO:teuthology.orchestra.run.smithi192.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 378 kB/s | 25 kB 00:00 2024-05-03T17:41:49.234 INFO:teuthology.orchestra.run.smithi122.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 301 kB/s | 253 kB 00:00 2024-05-03T17:41:49.418 INFO:teuthology.orchestra.run.smithi122.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 149 kB/s | 106 kB 00:00 2024-05-03T17:41:49.487 INFO:teuthology.orchestra.run.smithi192.stdout:(10/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el 13 MB/s | 11 MB 00:00 2024-05-03T17:41:49.587 INFO:teuthology.orchestra.run.smithi192.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 2.2 MB/s | 225 kB 00:00 2024-05-03T17:41:49.668 INFO:teuthology.orchestra.run.smithi122.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-05-03T17:41:49.812 INFO:teuthology.orchestra.run.smithi192.stdout:(12/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x8 7.8 MB/s | 17 MB 00:02 2024-05-03T17:41:49.838 INFO:teuthology.orchestra.run.smithi192.stdout:(13/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 108 kB/s | 27 kB 00:00 2024-05-03T17:41:49.871 INFO:teuthology.orchestra.run.smithi192.stdout:(14/152): libcephfs-devel-19.0.0-3450.g9a047fe7 542 kB/s | 32 kB 00:00 2024-05-03T17:41:49.921 INFO:teuthology.orchestra.run.smithi192.stdout:(15/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 8.7 MB/s | 747 kB 00:00 2024-05-03T17:41:49.947 INFO:teuthology.orchestra.run.smithi192.stdout:(16/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 2.1 MB/s | 163 kB 00:00 2024-05-03T17:41:49.981 INFO:teuthology.orchestra.run.smithi192.stdout:(17/152): librados-devel-19.0.0-3450.g9a047fe7. 2.1 MB/s | 127 kB 00:00 2024-05-03T17:41:50.018 INFO:teuthology.orchestra.run.smithi122.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 172 kB/s | 135 kB 00:00 2024-05-03T17:41:50.031 INFO:teuthology.orchestra.run.smithi192.stdout:(18/152): libradosstriper1-19.0.0-3450.g9a047fe 5.8 MB/s | 500 kB 00:00 2024-05-03T17:41:50.210 INFO:teuthology.orchestra.run.smithi122.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 158 kB/s | 125 kB 00:00 2024-05-03T17:41:50.298 INFO:teuthology.orchestra.run.smithi192.stdout:(19/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 17 MB/s | 5.4 MB 00:00 2024-05-03T17:41:50.311 INFO:teuthology.orchestra.run.smithi122.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-05-03T17:41:50.348 INFO:teuthology.orchestra.run.smithi192.stdout:(20/152): python3-ceph-argparse-19.0.0-3450.g9a 142 kB/s | 45 kB 00:00 2024-05-03T17:41:50.373 INFO:teuthology.orchestra.run.smithi192.stdout:(21/152): python3-ceph-common-19.0.0-3450.g9a04 1.8 MB/s | 135 kB 00:00 2024-05-03T17:41:50.377 INFO:teuthology.orchestra.run.smithi122.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-03T17:41:50.407 INFO:teuthology.orchestra.run.smithi192.stdout:(22/152): python3-cephfs-19.0.0-3450.g9a047fe7. 2.7 MB/s | 161 kB 00:00 2024-05-03T17:41:50.436 INFO:teuthology.orchestra.run.smithi122.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 368 kB/s | 21 kB 00:00 2024-05-03T17:41:50.441 INFO:teuthology.orchestra.run.smithi192.stdout:(23/152): python3-rados-19.0.0-3450.g9a047fe7.e 4.7 MB/s | 320 kB 00:00 2024-05-03T17:41:50.474 INFO:teuthology.orchestra.run.smithi192.stdout:(24/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 4.3 MB/s | 298 kB 00:00 2024-05-03T17:41:50.486 INFO:teuthology.orchestra.run.smithi122.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-05-03T17:41:50.500 INFO:teuthology.orchestra.run.smithi192.stdout:(25/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 1.7 MB/s | 100 kB 00:00 2024-05-03T17:41:50.503 INFO:teuthology.orchestra.run.smithi122.stdout:(52/152): python3-urllib3-1.26.5-5.el9.noarch.r 265 kB/s | 215 kB 00:00 2024-05-03T17:41:50.533 INFO:teuthology.orchestra.run.smithi192.stdout:(26/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-03T17:41:50.537 INFO:teuthology.orchestra.run.smithi122.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-05-03T17:41:50.562 INFO:teuthology.orchestra.run.smithi122.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 432 kB/s | 33 kB 00:00 2024-05-03T17:41:50.604 INFO:teuthology.orchestra.run.smithi122.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 271 kB/s | 18 kB 00:00 2024-05-03T17:41:50.633 INFO:teuthology.orchestra.run.smithi192.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 1.7 MB/s | 170 kB 00:00 2024-05-03T17:41:50.646 INFO:teuthology.orchestra.run.smithi122.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-05-03T17:41:50.697 INFO:teuthology.orchestra.run.smithi122.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-05-03T17:41:50.709 INFO:teuthology.orchestra.run.smithi192.stdout:(28/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 15 MB/s | 3.1 MB 00:00 2024-05-03T17:41:50.722 INFO:teuthology.orchestra.run.smithi122.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-05-03T17:41:50.734 INFO:teuthology.orchestra.run.smithi192.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3450.g 340 kB/s | 34 kB 00:00 2024-05-03T17:41:50.739 INFO:teuthology.orchestra.run.smithi122.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 491 kB/s | 45 kB 00:00 2024-05-03T17:41:50.781 INFO:teuthology.orchestra.run.smithi122.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-05-03T17:41:50.785 INFO:teuthology.orchestra.run.smithi192.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 1.9 MB/s | 142 kB 00:00 2024-05-03T17:41:50.823 INFO:teuthology.orchestra.run.smithi122.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 5.6 MB/s | 582 kB 00:00 2024-05-03T17:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 232 kB/s | 25 kB 00:00 2024-05-03T17:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 512 kB/s | 47 kB 00:00 2024-05-03T17:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-05-03T17:41:50.924 INFO:teuthology.orchestra.run.smithi122.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-05-03T17:41:50.950 INFO:teuthology.orchestra.run.smithi122.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 466 kB/s | 35 kB 00:00 2024-05-03T17:41:51.067 INFO:teuthology.orchestra.run.smithi122.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-05-03T17:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-03T17:41:51.176 INFO:teuthology.orchestra.run.smithi192.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 8.0 MB/s | 3.5 MB 00:00 2024-05-03T17:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-05-03T17:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-03T17:41:51.260 INFO:teuthology.orchestra.run.smithi192.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 3.2 MB/s | 274 kB 00:00 2024-05-03T17:41:51.344 INFO:teuthology.orchestra.run.smithi192.stdout:(33/152): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-05-03T17:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-05-03T17:41:51.369 INFO:teuthology.orchestra.run.smithi192.stdout:(34/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 454 kB/s | 49 kB 00:00 2024-05-03T17:41:51.403 INFO:teuthology.orchestra.run.smithi192.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3450.g9 281 kB/s | 16 kB 00:00 2024-05-03T17:41:51.436 INFO:teuthology.orchestra.run.smithi192.stdout:(36/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 4.0 MB/s | 273 kB 00:00 2024-05-03T17:41:51.452 INFO:teuthology.orchestra.run.smithi122.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-05-03T17:41:51.487 INFO:teuthology.orchestra.run.smithi192.stdout:(37/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 8.9 MB/s | 763 kB 00:00 2024-05-03T17:41:51.494 INFO:teuthology.orchestra.run.smithi122.stdout:(73/152): postgresql-13.14-1.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2024-05-03T17:41:51.503 INFO:teuthology.orchestra.run.smithi192.stdout:(38/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 272 kB/s | 18 kB 00:00 2024-05-03T17:41:51.510 INFO:teuthology.orchestra.run.smithi122.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 580 kB/s | 87 kB 00:00 2024-05-03T17:41:51.570 INFO:teuthology.orchestra.run.smithi122.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-05-03T17:41:51.612 INFO:teuthology.orchestra.run.smithi122.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-03T17:41:51.703 INFO:teuthology.orchestra.run.smithi122.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-05-03T17:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-05-03T17:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout:(79/152): python3-jsonpatch-1.21-16.el9.noarch. 139 kB/s | 26 kB 00:00 2024-05-03T17:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-03T17:41:51.804 INFO:teuthology.orchestra.run.smithi122.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 459 kB/s | 35 kB 00:00 2024-05-03T17:41:51.855 INFO:teuthology.orchestra.run.smithi122.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-05-03T17:41:51.880 INFO:teuthology.orchestra.run.smithi122.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-05-03T17:41:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 5.1 MB 00:00 2024-05-03T17:41:51.973 INFO:teuthology.orchestra.run.smithi122.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 657 kB/s | 77 kB 00:00 2024-05-03T17:41:51.997 INFO:teuthology.orchestra.run.smithi122.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 360 kB/s | 42 kB 00:00 2024-05-03T17:41:52.032 INFO:teuthology.orchestra.run.smithi122.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-05-03T17:41:52.065 INFO:teuthology.orchestra.run.smithi122.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-03T17:41:52.108 INFO:teuthology.orchestra.run.smithi122.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-05-03T17:41:52.133 INFO:teuthology.orchestra.run.smithi122.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 506 kB/s | 51 kB 00:00 2024-05-03T17:41:52.158 INFO:teuthology.orchestra.run.smithi122.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 581 kB/s | 54 kB 00:00 2024-05-03T17:41:52.161 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:18 2024-05-03T17:41:52.196 INFO:teuthology.orchestra.run.smithi192.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 113 kB/s | 80 kB 00:00 2024-05-03T17:41:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-05-03T17:41:52.233 INFO:teuthology.orchestra.run.smithi122.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 588 kB/s | 44 kB 00:00 2024-05-03T17:41:52.275 INFO:teuthology.orchestra.run.smithi122.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-05-03T17:41:52.309 INFO:teuthology.orchestra.run.smithi122.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 475 kB/s | 36 kB 00:00 2024-05-03T17:41:52.351 INFO:teuthology.orchestra.run.smithi122.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-05-03T17:41:52.377 INFO:teuthology.orchestra.run.smithi122.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-05-03T17:41:52.419 INFO:teuthology.orchestra.run.smithi122.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 861 kB/s | 58 kB 00:00 2024-05-03T17:41:52.444 INFO:teuthology.orchestra.run.smithi122.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-05-03T17:41:52.496 INFO:teuthology.orchestra.run.smithi192.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 809 kB/s | 803 kB 00:00 2024-05-03T17:41:52.619 INFO:teuthology.orchestra.run.smithi122.stdout:(100/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-05-03T17:41:52.645 INFO:teuthology.orchestra.run.smithi122.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 282 kB/s | 64 kB 00:00 2024-05-03T17:41:52.670 INFO:teuthology.orchestra.run.smithi122.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 198 kB/s | 45 kB 00:00 2024-05-03T17:41:52.720 INFO:teuthology.orchestra.run.smithi122.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-05-03T17:41:52.962 INFO:teuthology.orchestra.run.smithi122.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-05-03T17:41:52.980 INFO:teuthology.orchestra.run.smithi192.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 249 kB/s | 195 kB 00:00 2024-05-03T17:41:52.987 INFO:teuthology.orchestra.run.smithi122.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-05-03T17:41:53.013 INFO:teuthology.orchestra.run.smithi122.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-05-03T17:41:53.096 INFO:teuthology.orchestra.run.smithi122.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-05-03T17:41:53.122 INFO:teuthology.orchestra.run.smithi122.stdout:(108/152): grpc-data-1.46.7-10.el9.noarch.rpm 122 kB/s | 19 kB 00:00 2024-05-03T17:41:53.222 INFO:teuthology.orchestra.run.smithi122.stdout:(109/152): libarrow-9.0.0-11.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-05-03T17:41:53.247 INFO:teuthology.orchestra.run.smithi122.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 322 kB/s | 49 kB 00:00 2024-05-03T17:41:53.281 INFO:teuthology.orchestra.run.smithi122.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 424 kB/s | 67 kB 00:00 2024-05-03T17:41:53.307 INFO:teuthology.orchestra.run.smithi122.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-05-03T17:41:53.313 INFO:teuthology.orchestra.run.smithi192.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 310 kB/s | 253 kB 00:00 2024-05-03T17:41:53.382 INFO:teuthology.orchestra.run.smithi122.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-03T17:41:53.416 INFO:teuthology.orchestra.run.smithi122.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-05-03T17:41:53.483 INFO:teuthology.orchestra.run.smithi122.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 320 kB/s | 32 kB 00:00 2024-05-03T17:41:53.508 INFO:teuthology.orchestra.run.smithi122.stdout:(116/152): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-05-03T17:41:53.566 INFO:teuthology.orchestra.run.smithi122.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 99 kB/s | 15 kB 00:00 2024-05-03T17:41:53.608 INFO:teuthology.orchestra.run.smithi122.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-03T17:41:53.684 INFO:teuthology.orchestra.run.smithi122.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-05-03T17:41:53.701 INFO:teuthology.orchestra.run.smithi122.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-05-03T17:41:53.726 INFO:teuthology.orchestra.run.smithi122.stdout:(121/152): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-05-03T17:41:53.759 INFO:teuthology.orchestra.run.smithi122.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-03T17:41:53.810 INFO:teuthology.orchestra.run.smithi122.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-05-03T17:41:53.835 INFO:teuthology.orchestra.run.smithi122.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-05-03T17:41:53.894 INFO:teuthology.orchestra.run.smithi122.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-05-03T17:41:53.919 INFO:teuthology.orchestra.run.smithi122.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-05-03T17:41:53.953 INFO:teuthology.orchestra.run.smithi122.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 197 kB/s | 23 kB 00:00 2024-05-03T17:41:53.978 INFO:teuthology.orchestra.run.smithi122.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-05-03T17:41:53.988 INFO:teuthology.orchestra.run.smithi192.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-05-03T17:41:54.003 INFO:teuthology.orchestra.run.smithi122.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-05-03T17:41:54.045 INFO:teuthology.orchestra.run.smithi122.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-05-03T17:41:54.058 INFO:teuthology.orchestra.run.smithi192.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 143 kB/s | 106 kB 00:00 2024-05-03T17:41:54.070 INFO:teuthology.orchestra.run.smithi122.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-05-03T17:41:54.096 INFO:teuthology.orchestra.run.smithi122.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 855 kB/s | 79 kB 00:00 2024-05-03T17:41:54.121 INFO:teuthology.orchestra.run.smithi122.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 764 kB/s | 58 kB 00:00 2024-05-03T17:41:54.146 INFO:teuthology.orchestra.run.smithi122.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-03T17:41:54.172 INFO:teuthology.orchestra.run.smithi122.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-05-03T17:41:54.197 INFO:teuthology.orchestra.run.smithi122.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 560 kB/s | 28 kB 00:00 2024-05-03T17:41:54.223 INFO:teuthology.orchestra.run.smithi122.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 890 kB/s | 90 kB 00:00 2024-05-03T17:41:54.248 INFO:teuthology.orchestra.run.smithi122.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-05-03T17:41:54.282 INFO:teuthology.orchestra.run.smithi122.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-03T17:41:54.307 INFO:teuthology.orchestra.run.smithi122.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 703 kB/s | 59 kB 00:00 2024-05-03T17:41:54.332 INFO:teuthology.orchestra.run.smithi122.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 425 kB/s | 36 kB 00:00 2024-05-03T17:41:54.358 INFO:teuthology.orchestra.run.smithi122.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-03T17:41:54.391 INFO:teuthology.orchestra.run.smithi122.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-03T17:41:54.434 INFO:teuthology.orchestra.run.smithi122.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-05-03T17:41:54.458 INFO:teuthology.orchestra.run.smithi122.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-05-03T17:41:54.484 INFO:teuthology.orchestra.run.smithi122.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-03T17:41:54.509 INFO:teuthology.orchestra.run.smithi122.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-03T17:41:54.568 INFO:teuthology.orchestra.run.smithi122.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-03T17:41:54.898 INFO:teuthology.orchestra.run.smithi192.stdout:(45/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 8.1 MB/s | 49 MB 00:06 2024-05-03T17:41:54.918 INFO:teuthology.orchestra.run.smithi122.stdout:(149/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 7.7 MB/s | 3.1 MB 00:00 2024-05-03T17:41:55.035 INFO:teuthology.orchestra.run.smithi122.stdout:(150/152): librados2-19.0.0-3450.g9a047fe7.el9. 6.2 MB/s | 3.4 MB 00:00 2024-05-03T17:41:55.073 INFO:teuthology.orchestra.run.smithi192.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 125 kB/s | 135 kB 00:01 2024-05-03T17:41:55.098 INFO:teuthology.orchestra.run.smithi192.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 120 kB/s | 125 kB 00:01 2024-05-03T17:41:55.357 INFO:teuthology.orchestra.run.smithi192.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-05-03T17:41:55.424 INFO:teuthology.orchestra.run.smithi192.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-05-03T17:41:55.449 INFO:teuthology.orchestra.run.smithi192.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-05-03T17:41:55.474 INFO:teuthology.orchestra.run.smithi192.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-05-03T17:41:55.524 INFO:teuthology.orchestra.run.smithi192.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-05-03T17:41:55.574 INFO:teuthology.orchestra.run.smithi192.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-05-03T17:41:55.625 INFO:teuthology.orchestra.run.smithi192.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-05-03T17:41:55.644 INFO:teuthology.orchestra.run.smithi122.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-05-03T17:41:55.675 INFO:teuthology.orchestra.run.smithi192.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-05-03T17:41:55.725 INFO:teuthology.orchestra.run.smithi192.stdout:(56/152): python3-urllib3-1.26.5-5.el9.noarch.r 259 kB/s | 215 kB 00:00 2024-05-03T17:41:55.776 INFO:teuthology.orchestra.run.smithi192.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-05-03T17:41:55.859 INFO:teuthology.orchestra.run.smithi192.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-05-03T17:41:55.968 INFO:teuthology.orchestra.run.smithi192.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-05-03T17:41:56.019 INFO:teuthology.orchestra.run.smithi122.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-05-03T17:41:56.024 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-05-03T17:41:56.025 INFO:teuthology.orchestra.run.smithi122.stdout:Total 15 MB/s | 222 MB 00:14 2024-05-03T17:41:56.026 INFO:teuthology.orchestra.run.smithi192.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-05-03T17:41:56.052 INFO:teuthology.orchestra.run.smithi192.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 3.0 MB/s | 582 kB 00:00 2024-05-03T17:41:56.077 INFO:teuthology.orchestra.run.smithi192.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-05-03T17:41:56.161 INFO:teuthology.orchestra.run.smithi192.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-03T17:41:56.244 INFO:teuthology.orchestra.run.smithi192.stdout:(64/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.2 MB/s | 3.0 MB 00:00 2024-05-03T17:41:56.270 INFO:teuthology.orchestra.run.smithi192.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.3 MB/s | 292 kB 00:00 2024-05-03T17:41:56.295 INFO:teuthology.orchestra.run.smithi192.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 263 kB/s | 35 kB 00:00 2024-05-03T17:41:56.620 INFO:teuthology.orchestra.run.smithi192.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-05-03T17:41:56.837 INFO:teuthology.orchestra.run.smithi192.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 4.1 MB/s | 911 kB 00:00 2024-05-03T17:41:56.896 INFO:teuthology.orchestra.run.smithi192.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-03T17:41:57.080 INFO:teuthology.orchestra.run.smithi192.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.7 MB/s | 4.7 MB 00:00 2024-05-03T17:41:57.136 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-05-03T17:41:57.250 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-03T17:41:57.255 INFO:teuthology.orchestra.run.smithi192.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-05-03T17:41:57.278 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-05-03T17:41:57.278 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-05-03T17:41:57.381 INFO:teuthology.orchestra.run.smithi192.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2024-05-03T17:41:57.497 INFO:teuthology.orchestra.run.smithi192.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 361 kB/s | 87 kB 00:00 2024-05-03T17:41:57.597 INFO:teuthology.orchestra.run.smithi192.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-03T17:41:57.681 INFO:teuthology.orchestra.run.smithi192.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-03T17:41:57.740 INFO:teuthology.orchestra.run.smithi192.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-05-03T17:41:57.798 INFO:teuthology.orchestra.run.smithi192.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-05-03T17:41:57.849 INFO:teuthology.orchestra.run.smithi192.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-03T17:41:57.899 INFO:teuthology.orchestra.run.smithi192.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-05-03T17:41:58.024 INFO:teuthology.orchestra.run.smithi192.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 5.1 MB/s | 5.8 MB 00:01 2024-05-03T17:41:58.082 INFO:teuthology.orchestra.run.smithi192.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-05-03T17:41:58.141 INFO:teuthology.orchestra.run.smithi192.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-03T17:41:58.200 INFO:teuthology.orchestra.run.smithi192.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-03T17:41:58.259 INFO:teuthology.orchestra.run.smithi192.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 725 kB/s | 42 kB 00:00 2024-05-03T17:41:58.317 INFO:teuthology.orchestra.run.smithi192.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-03T17:41:58.376 INFO:teuthology.orchestra.run.smithi192.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-03T17:41:58.452 INFO:teuthology.orchestra.run.smithi192.stdout:(87/152): python3-babel-2.9.1-2.el9.noarch.rpm 5.6 MB/s | 6.0 MB 00:01 2024-05-03T17:41:58.477 INFO:teuthology.orchestra.run.smithi192.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-05-03T17:41:58.502 INFO:teuthology.orchestra.run.smithi192.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-03T17:41:58.535 INFO:teuthology.orchestra.run.smithi192.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 919 kB/s | 54 kB 00:00 2024-05-03T17:41:58.602 INFO:teuthology.orchestra.run.smithi192.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 625 kB/s | 42 kB 00:00 2024-05-03T17:41:58.661 INFO:teuthology.orchestra.run.smithi192.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-05-03T17:41:58.720 INFO:teuthology.orchestra.run.smithi192.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-03T17:41:58.820 INFO:teuthology.orchestra.run.smithi192.stdout:(94/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.5 MB/s | 5.1 MB 00:00 2024-05-03T17:41:58.845 INFO:teuthology.orchestra.run.smithi192.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 286 kB/s | 36 kB 00:00 2024-05-03T17:41:58.879 INFO:teuthology.orchestra.run.smithi192.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-05-03T17:41:58.904 INFO:teuthology.orchestra.run.smithi192.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-05-03T17:41:58.929 INFO:teuthology.orchestra.run.smithi192.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-03T17:41:58.955 INFO:teuthology.orchestra.run.smithi192.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-03T17:41:58.980 INFO:teuthology.orchestra.run.smithi192.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-03T17:41:59.006 INFO:teuthology.orchestra.run.smithi192.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-05-03T17:41:59.031 INFO:teuthology.orchestra.run.smithi192.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 442 kB/s | 22 kB 00:00 2024-05-03T17:41:59.123 INFO:teuthology.orchestra.run.smithi192.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.2 MB/s | 863 kB 00:00 2024-05-03T17:41:59.248 INFO:teuthology.orchestra.run.smithi192.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.5 MB/s | 548 kB 00:00 2024-05-03T17:41:59.298 INFO:teuthology.orchestra.run.smithi192.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 391 kB/s | 19 kB 00:00 2024-05-03T17:41:59.303 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-05-03T17:41:59.304 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-05-03T17:41:59.324 INFO:teuthology.orchestra.run.smithi192.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-05-03T17:41:59.374 INFO:teuthology.orchestra.run.smithi192.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-05-03T17:41:59.401 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T17:41:59.402 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T17:41:59.412 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T17:41:59.413 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T17:41:59.414 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T17:41:59.415 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T17:41:59.417 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T17:41:59.419 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T17:41:59.420 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T17:41:59.424 INFO:teuthology.orchestra.run.smithi192.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 978 kB/s | 49 kB 00:00 2024-05-03T17:41:59.481 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-03T17:41:59.499 INFO:teuthology.orchestra.run.smithi192.stdout:(109/152): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-05-03T17:41:59.501 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-03T17:41:59.501 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-03T17:41:59.501 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-03T17:41:59.501 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-03T17:41:59.501 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T17:41:59.501 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T17:41:59.501 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T17:41:59.501 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T17:41:59.502 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T17:41:59.503 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T17:41:59.504 INFO:teuthology.orchestra.run.smithi149.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T17:41:59.505 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T17:41:59.506 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T17:41:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T17:41:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T17:41:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T17:41:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T17:41:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T17:41:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T17:41:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T17:41:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T17:41:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T17:41:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T17:41:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T17:41:59.508 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T17:41:59.509 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T17:41:59.510 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T17:41:59.511 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T17:41:59.512 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T17:41:59.513 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:41:59.514 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-03T17:41:59.514 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-03T17:41:59.514 INFO:teuthology.orchestra.run.smithi149.stdout:Install 148 Packages 2024-05-03T17:41:59.514 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 4 Packages 2024-05-03T17:41:59.514 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:41:59.514 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 222 M 2024-05-03T17:41:59.515 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-03T17:41:59.525 INFO:teuthology.orchestra.run.smithi192.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-05-03T17:41:59.550 INFO:teuthology.orchestra.run.smithi192.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-05-03T17:41:59.601 INFO:teuthology.orchestra.run.smithi192.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-05-03T17:41:59.626 INFO:teuthology.orchestra.run.smithi192.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-05-03T17:41:59.659 INFO:teuthology.orchestra.run.smithi192.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 740 kB/s | 43 kB 00:00 2024-05-03T17:41:59.685 INFO:teuthology.orchestra.run.smithi192.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-05-03T17:41:59.701 INFO:teuthology.orchestra.run.smithi192.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 359 kB/s | 15 kB 00:00 2024-05-03T17:41:59.744 INFO:teuthology.orchestra.run.smithi192.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-03T17:41:59.769 INFO:teuthology.orchestra.run.smithi192.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-05-03T17:41:59.794 INFO:teuthology.orchestra.run.smithi192.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-05-03T17:41:59.820 INFO:teuthology.orchestra.run.smithi192.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 706 kB/s | 35 kB 00:00 2024-05-03T17:41:59.864 INFO:teuthology.orchestra.run.smithi192.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.9 MB/s | 206 kB 00:00 2024-05-03T17:41:59.912 INFO:teuthology.orchestra.run.smithi192.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 3.0 MB/s | 144 kB 00:00 2024-05-03T17:41:59.962 INFO:teuthology.orchestra.run.smithi192.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-03T17:41:59.987 INFO:teuthology.orchestra.run.smithi192.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-05-03T17:42:00.013 INFO:teuthology.orchestra.run.smithi192.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-05-03T17:42:00.038 INFO:teuthology.orchestra.run.smithi192.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-05-03T17:42:00.063 INFO:teuthology.orchestra.run.smithi192.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 384 kB/s | 19 kB 00:00 2024-05-03T17:42:00.147 INFO:teuthology.orchestra.run.smithi192.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-05-03T17:42:00.189 INFO:teuthology.orchestra.run.smithi192.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-05-03T17:42:00.214 INFO:teuthology.orchestra.run.smithi192.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-05-03T17:42:00.240 INFO:teuthology.orchestra.run.smithi192.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-03T17:42:00.243 INFO:teuthology.orchestra.run.smithi149.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-05-03T17:42:00.265 INFO:teuthology.orchestra.run.smithi192.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-03T17:42:00.290 INFO:teuthology.orchestra.run.smithi192.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-05-03T17:42:00.316 INFO:teuthology.orchestra.run.smithi192.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-05-03T17:42:00.341 INFO:teuthology.orchestra.run.smithi192.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-03T17:42:00.391 INFO:teuthology.orchestra.run.smithi192.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-05-03T17:42:00.417 INFO:teuthology.orchestra.run.smithi192.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-05-03T17:42:00.458 INFO:teuthology.orchestra.run.smithi192.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-05-03T17:42:00.509 INFO:teuthology.orchestra.run.smithi192.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-05-03T17:42:00.534 INFO:teuthology.orchestra.run.smithi192.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-05-03T17:42:00.559 INFO:teuthology.orchestra.run.smithi149.stdout:(2/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 9.8 MB/s | 5.4 MB 00:00 2024-05-03T17:42:00.610 INFO:teuthology.orchestra.run.smithi149.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 2.8 MB/s | 145 kB 00:00 2024-05-03T17:42:00.634 INFO:teuthology.orchestra.run.smithi192.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 407 kB/s | 51 kB 00:00 2024-05-03T17:42:00.660 INFO:teuthology.orchestra.run.smithi192.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-05-03T17:42:00.685 INFO:teuthology.orchestra.run.smithi192.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-03T17:42:00.718 INFO:teuthology.orchestra.run.smithi192.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-05-03T17:42:00.735 INFO:teuthology.orchestra.run.smithi192.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-05-03T17:42:00.769 INFO:teuthology.orchestra.run.smithi149.stdout:(4/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 15 MB/s | 2.4 MB 00:00 2024-05-03T17:42:00.769 INFO:teuthology.orchestra.run.smithi192.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-03T17:42:00.819 INFO:teuthology.orchestra.run.smithi149.stdout:(5/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.5 MB/s | 891 kB 00:00 2024-05-03T17:42:00.828 INFO:teuthology.orchestra.run.smithi192.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-03T17:42:00.869 INFO:teuthology.orchestra.run.smithi149.stdout:(6/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 15 MB/s | 1.5 MB 00:00 2024-05-03T17:42:01.803 INFO:teuthology.orchestra.run.smithi192.stdout:(148/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.9 MB/s | 16 MB 00:03 2024-05-03T17:42:01.853 INFO:teuthology.orchestra.run.smithi192.stdout:(149/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-05-03T17:42:01.912 INFO:teuthology.orchestra.run.smithi192.stdout:(150/152): librados2-19.0.0-3450.g9a047fe7.el9. 3.0 MB/s | 3.4 MB 00:01 2024-05-03T17:42:02.011 INFO:teuthology.orchestra.run.smithi149.stdout:(7/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x86 15 MB/s | 17 MB 00:01 2024-05-03T17:42:02.745 INFO:teuthology.orchestra.run.smithi149.stdout:(8/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el9 14 MB/s | 11 MB 00:00 2024-05-03T17:42:02.837 INFO:teuthology.orchestra.run.smithi192.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-05-03T17:42:02.895 INFO:teuthology.orchestra.run.smithi149.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 168 kB/s | 25 kB 00:00 2024-05-03T17:42:02.962 INFO:teuthology.orchestra.run.smithi192.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-05-03T17:42:02.966 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-03T17:42:02.966 INFO:teuthology.orchestra.run.smithi192.stdout:Total 12 MB/s | 222 MB 00:18 2024-05-03T17:42:03.154 INFO:teuthology.orchestra.run.smithi149.stdout:(10/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x8 2.0 MB/s | 4.7 MB 00:02 2024-05-03T17:42:03.312 INFO:teuthology.orchestra.run.smithi149.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 1.4 MB/s | 225 kB 00:00 2024-05-03T17:42:03.404 INFO:teuthology.orchestra.run.smithi149.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 295 kB/s | 27 kB 00:00 2024-05-03T17:42:03.496 INFO:teuthology.orchestra.run.smithi149.stdout:(13/152): libcephfs-devel-19.0.0-3450.g9a047fe7 347 kB/s | 32 kB 00:00 2024-05-03T17:42:03.855 INFO:teuthology.orchestra.run.smithi149.stdout:(14/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 2.0 MB/s | 747 kB 00:00 2024-05-03T17:42:03.955 INFO:teuthology.orchestra.run.smithi149.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 1.6 MB/s | 163 kB 00:00 2024-05-03T17:42:04.029 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-03T17:42:04.047 INFO:teuthology.orchestra.run.smithi149.stdout:(16/152): librados-devel-19.0.0-3450.g9a047fe7. 1.4 MB/s | 127 kB 00:00 2024-05-03T17:42:04.171 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-03T17:42:04.171 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-03T17:42:04.339 INFO:teuthology.orchestra.run.smithi149.stdout:(17/152): libradosstriper1-19.0.0-3450.g9a047fe 1.7 MB/s | 500 kB 00:00 2024-05-03T17:42:05.337 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-05-03T17:42:05.464 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T17:42:05.481 INFO:teuthology.orchestra.run.smithi149.stdout:(18/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 19 MB/s | 49 MB 00:02 2024-05-03T17:42:05.522 INFO:teuthology.orchestra.run.smithi149.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 1.1 MB/s | 45 kB 00:00 2024-05-03T17:42:05.573 INFO:teuthology.orchestra.run.smithi149.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 2.7 MB/s | 135 kB 00:00 2024-05-03T17:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T17:42:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T17:42:05.781 INFO:teuthology.orchestra.run.smithi149.stdout:(21/152): ceph-common-19.0.0-3450.g9a047fe7.el9 3.7 MB/s | 21 MB 00:05 2024-05-03T17:42:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T17:42:05.807 INFO:teuthology.orchestra.run.smithi149.stdout:(22/152): python3-cephfs-19.0.0-3450.g9a047fe7. 691 kB/s | 161 kB 00:00 2024-05-03T17:42:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T17:42:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T17:42:05.857 INFO:teuthology.orchestra.run.smithi149.stdout:(23/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 5.9 MB/s | 298 kB 00:00 2024-05-03T17:42:05.882 INFO:teuthology.orchestra.run.smithi149.stdout:(24/152): python3-rados-19.0.0-3450.g9a047fe7.e 3.1 MB/s | 320 kB 00:00 2024-05-03T17:42:05.907 INFO:teuthology.orchestra.run.smithi149.stdout:(25/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 1.9 MB/s | 100 kB 00:00 2024-05-03T17:42:05.941 INFO:teuthology.orchestra.run.smithi149.stdout:(26/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-03T17:42:05.942 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-03T17:42:05.942 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-03T17:42:06.033 INFO:teuthology.orchestra.run.smithi149.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 1.8 MB/s | 170 kB 00:00 2024-05-03T17:42:06.108 INFO:teuthology.orchestra.run.smithi149.stdout:(28/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 15 MB/s | 3.1 MB 00:00 2024-05-03T17:42:06.134 INFO:teuthology.orchestra.run.smithi149.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3450.g 341 kB/s | 34 kB 00:00 2024-05-03T17:42:06.159 INFO:teuthology.orchestra.run.smithi149.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 2.8 MB/s | 142 kB 00:00 2024-05-03T17:42:06.181 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T17:42:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T17:42:06.415 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T17:42:06.426 INFO:teuthology.orchestra.run.smithi149.stdout:(31/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 2.6 MB/s | 5.4 MB 00:02 2024-05-03T17:42:06.444 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T17:42:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T17:42:06.568 INFO:teuthology.orchestra.run.smithi149.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 18 MB/s | 7.4 MB 00:00 2024-05-03T17:42:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T17:42:06.593 INFO:teuthology.orchestra.run.smithi149.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3450.g9a 1.6 MB/s | 274 kB 00:00 2024-05-03T17:42:06.619 INFO:teuthology.orchestra.run.smithi149.stdout:(34/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 984 kB/s | 49 kB 00:00 2024-05-03T17:42:06.652 INFO:teuthology.orchestra.run.smithi149.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3450.g9 282 kB/s | 16 kB 00:00 2024-05-03T17:42:06.677 INFO:teuthology.orchestra.run.smithi149.stdout:(36/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 4.6 MB/s | 273 kB 00:00 2024-05-03T17:42:06.719 INFO:teuthology.orchestra.run.smithi149.stdout:(37/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 440 kB/s | 18 kB 00:00 2024-05-03T17:42:06.729 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T17:42:06.779 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T17:42:06.808 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T17:42:06.861 INFO:teuthology.orchestra.run.smithi149.stdout:(38/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 4.9 MB/s | 3.5 MB 00:00 2024-05-03T17:42:06.887 INFO:teuthology.orchestra.run.smithi149.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 481 kB/s | 80 kB 00:00 2024-05-03T17:42:06.911 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T17:42:06.911 INFO:teuthology.orchestra.run.smithi149.stdout:(40/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 2.9 MB/s | 763 kB 00:00 2024-05-03T17:42:06.934 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T17:42:06.995 INFO:teuthology.orchestra.run.smithi149.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-05-03T17:42:06.996 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T17:42:07.018 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T17:42:07.037 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T17:42:07.112 INFO:teuthology.orchestra.run.smithi149.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-05-03T17:42:07.138 INFO:teuthology.orchestra.run.smithi149.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-05-03T17:42:07.163 INFO:teuthology.orchestra.run.smithi149.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-03T17:42:07.197 INFO:teuthology.orchestra.run.smithi149.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-03T17:42:07.230 INFO:teuthology.orchestra.run.smithi149.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-05-03T17:42:07.235 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T17:42:07.255 INFO:teuthology.orchestra.run.smithi149.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-05-03T17:42:07.268 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T17:42:07.281 INFO:teuthology.orchestra.run.smithi149.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-03T17:42:07.384 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T17:42:07.482 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T17:42:07.519 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T17:42:07.554 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T17:42:07.628 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T17:42:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T17:42:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T17:42:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T17:42:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T17:42:07.854 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T17:42:07.931 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T17:42:07.969 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T17:42:08.013 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T17:42:08.031 INFO:teuthology.orchestra.run.smithi149.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 142 kB/s | 107 kB 00:00 2024-05-03T17:42:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T17:42:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T17:42:08.159 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T17:42:08.187 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T17:42:08.243 INFO:teuthology.orchestra.run.smithi149.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 680 kB/s | 689 kB 00:01 2024-05-03T17:42:08.258 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T17:42:08.281 INFO:teuthology.orchestra.run.smithi149.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 919 kB/s | 943 kB 00:01 2024-05-03T17:42:08.303 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T17:42:08.378 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T17:42:08.514 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T17:42:08.564 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T17:42:08.715 INFO:teuthology.orchestra.run.smithi149.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 31 kB/s | 21 kB 00:00 2024-05-03T17:42:08.899 INFO:teuthology.orchestra.run.smithi149.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 53 kB/s | 33 kB 00:00 2024-05-03T17:42:09.096 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T17:42:09.115 INFO:teuthology.orchestra.run.smithi149.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 360 kB/s | 313 kB 00:00 2024-05-03T17:42:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T17:42:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T17:42:09.257 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T17:42:09.295 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T17:42:09.336 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T17:42:09.424 INFO:teuthology.orchestra.run.smithi149.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-05-03T17:42:09.481 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T17:42:09.531 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T17:42:09.586 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T17:42:09.658 INFO:teuthology.orchestra.run.smithi149.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 11 kB 00:00 2024-05-03T17:42:09.660 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T17:42:09.699 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T17:42:09.735 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T17:42:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T17:42:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T17:42:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T17:42:09.883 INFO:teuthology.orchestra.run.smithi149.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-05-03T17:42:09.903 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T17:42:10.040 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T17:42:10.084 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T17:42:10.116 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T17:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T17:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T17:42:10.266 INFO:teuthology.orchestra.run.smithi149.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 190 kB/s | 160 kB 00:00 2024-05-03T17:42:10.281 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T17:42:10.321 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T17:42:10.325 INFO:teuthology.orchestra.run.smithi149.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 68 kB/s | 45 kB 00:00 2024-05-03T17:42:10.347 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T17:42:10.372 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T17:42:10.448 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T17:42:10.575 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T17:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T17:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T17:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T17:42:10.723 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T17:42:10.755 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T17:42:10.785 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T17:42:10.820 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T17:42:10.853 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T17:42:10.892 INFO:teuthology.orchestra.run.smithi149.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 657 kB/s | 662 kB 00:01 2024-05-03T17:42:10.895 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T17:42:10.929 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T17:42:10.992 INFO:teuthology.orchestra.run.smithi149.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 38 kB/s | 25 kB 00:00 2024-05-03T17:42:11.017 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T17:42:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T17:42:11.300 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T17:42:11.301 INFO:teuthology.orchestra.run.smithi149.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 563 kB/s | 582 kB 00:01 2024-05-03T17:42:11.348 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T17:42:11.457 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T17:42:11.510 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T17:42:11.593 INFO:teuthology.orchestra.run.smithi149.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 68 kB/s | 47 kB 00:00 2024-05-03T17:42:11.868 INFO:teuthology.orchestra.run.smithi149.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 334 kB/s | 292 kB 00:00 2024-05-03T17:42:11.921 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-03T17:42:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T17:42:12.102 INFO:teuthology.orchestra.run.smithi149.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 235 kB/s | 188 kB 00:00 2024-05-03T17:42:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T17:42:12.193 INFO:teuthology.orchestra.run.smithi149.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 59 kB/s | 35 kB 00:00 2024-05-03T17:42:12.241 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T17:42:12.259 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T17:42:12.291 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T17:42:12.293 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T17:42:12.318 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T17:42:12.324 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T17:42:12.361 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T17:42:12.426 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T17:42:12.483 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T17:42:12.572 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T17:42:12.692 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T17:42:12.772 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T17:42:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T17:42:12.915 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T17:42:12.945 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T17:42:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T17:42:13.046 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T17:42:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T17:42:13.075 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T17:42:13.094 INFO:teuthology.orchestra.run.smithi149.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.9 MB/s | 4.7 MB 00:01 2024-05-03T17:42:13.152 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T17:42:13.228 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T17:42:13.235 INFO:teuthology.orchestra.run.smithi149.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-05-03T17:42:13.279 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T17:42:13.308 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T17:42:13.344 INFO:teuthology.orchestra.run.smithi149.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-05-03T17:42:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T17:42:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T17:42:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T17:42:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T17:42:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T17:42:13.667 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T17:42:13.702 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T17:42:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T17:42:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T17:42:13.905 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T17:42:13.925 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T17:42:13.934 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T17:42:13.969 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T17:42:14.034 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T17:42:14.061 INFO:teuthology.orchestra.run.smithi149.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 161 kB/s | 133 kB 00:00 2024-05-03T17:42:14.067 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T17:42:14.094 INFO:teuthology.orchestra.run.smithi149.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 911 kB/s | 911 kB 00:00 2024-05-03T17:42:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T17:42:14.144 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T17:42:14.171 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T17:42:14.241 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T17:42:14.294 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T17:42:14.324 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T17:42:14.360 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T17:42:14.437 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T17:42:14.479 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T17:42:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T17:42:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T17:42:14.595 INFO:teuthology.orchestra.run.smithi149.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-05-03T17:42:14.622 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T17:42:14.674 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T17:42:14.749 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T17:42:14.811 INFO:teuthology.orchestra.run.smithi149.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 122 kB/s | 87 kB 00:00 2024-05-03T17:42:14.894 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T17:42:14.951 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T17:42:15.104 INFO:teuthology.orchestra.run.smithi149.stdout:(74/152): protobuf-3.14.0-13.el9.x86_64.rpm 991 kB/s | 1.0 MB 00:01 2024-05-03T17:42:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T17:42:15.248 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T17:42:15.276 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T17:42:15.309 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T17:42:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T17:42:15.581 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T17:42:15.617 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T17:42:15.622 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T17:42:15.654 INFO:teuthology.orchestra.run.smithi149.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 290 kB/s | 244 kB 00:00 2024-05-03T17:42:15.660 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T17:42:15.672 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T17:42:15.698 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T17:42:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T17:42:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T17:42:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T17:42:15.812 INFO:teuthology.orchestra.run.smithi149.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 4.9 MB/s | 6.0 MB 00:01 2024-05-03T17:42:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T17:42:15.905 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T17:42:15.938 INFO:teuthology.orchestra.run.smithi149.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 297 kB/s | 248 kB 00:00 2024-05-03T17:42:15.968 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T17:42:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T17:42:16.096 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T17:42:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T17:42:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T17:42:16.210 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T17:42:16.246 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T17:42:16.280 INFO:teuthology.orchestra.run.smithi149.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 41 kB/s | 26 kB 00:00 2024-05-03T17:42:16.289 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T17:42:16.316 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T17:42:16.331 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T17:42:16.342 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T17:42:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T17:42:16.413 INFO:teuthology.orchestra.run.smithi149.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 32 kB/s | 19 kB 00:00 2024-05-03T17:42:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T17:42:16.469 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T17:42:16.511 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T17:42:16.541 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T17:42:16.583 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T17:42:16.679 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T17:42:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T17:42:16.730 INFO:teuthology.orchestra.run.smithi149.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 217 kB/s | 172 kB 00:00 2024-05-03T17:42:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T17:42:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T17:42:16.807 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T17:42:16.872 INFO:teuthology.orchestra.run.smithi149.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 59 kB/s | 35 kB 00:00 2024-05-03T17:42:16.882 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T17:42:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T17:42:17.047 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T17:42:17.075 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T17:42:17.110 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T17:42:17.149 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T17:42:17.180 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T17:42:17.210 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T17:42:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T17:42:17.362 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T17:42:17.404 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T17:42:17.437 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T17:42:17.526 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T17:42:17.539 INFO:teuthology.orchestra.run.smithi149.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 310 kB/s | 250 kB 00:00 2024-05-03T17:42:17.568 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T17:42:17.606 INFO:teuthology.orchestra.run.smithi149.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-05-03T17:42:17.607 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T17:42:17.654 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T17:42:17.689 INFO:teuthology.orchestra.run.smithi149.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 272 kB/s | 222 kB 00:00 2024-05-03T17:42:17.754 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T17:42:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T17:42:18.240 INFO:teuthology.orchestra.run.smithi149.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 110 kB/s | 77 kB 00:00 2024-05-03T17:42:18.265 INFO:teuthology.orchestra.run.smithi149.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 64 kB/s | 42 kB 00:00 2024-05-03T17:42:18.540 INFO:teuthology.orchestra.run.smithi149.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 317 kB/s | 269 kB 00:00 2024-05-03T17:42:18.573 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T17:42:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T17:42:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T17:42:18.738 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T17:42:18.754 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T17:42:18.774 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T17:42:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T17:42:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T17:42:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T17:42:18.990 INFO:teuthology.orchestra.run.smithi149.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 212 kB/s | 159 kB 00:00 2024-05-03T17:42:19.082 INFO:teuthology.orchestra.run.smithi149.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 341 kB/s | 279 kB 00:00 2024-05-03T17:42:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T17:42:19.191 INFO:teuthology.orchestra.run.smithi149.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 78 kB/s | 51 kB 00:00 2024-05-03T17:42:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T17:42:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T17:42:19.316 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T17:42:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T17:42:19.455 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T17:42:19.455 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T17:42:19.455 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T17:42:19.455 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:19.505 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T17:42:19.700 INFO:teuthology.orchestra.run.smithi149.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 76 kB/s | 54 kB 00:00 2024-05-03T17:42:19.808 INFO:teuthology.orchestra.run.smithi149.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-05-03T17:42:19.942 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T17:42:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T17:42:20.317 INFO:teuthology.orchestra.run.smithi149.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 72 kB/s | 44 kB 00:00 2024-05-03T17:42:20.650 INFO:teuthology.orchestra.run.smithi149.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-03T17:42:20.701 INFO:teuthology.orchestra.run.smithi149.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 247 kB/s | 221 kB 00:00 2024-05-03T17:42:21.051 INFO:teuthology.orchestra.run.smithi149.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 49 kB/s | 36 kB 00:00 2024-05-03T17:42:21.292 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T17:42:21.326 INFO:teuthology.orchestra.run.smithi149.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 73 kB/s | 47 kB 00:00 2024-05-03T17:42:21.372 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T17:42:21.400 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T17:42:21.433 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T17:42:21.526 INFO:teuthology.orchestra.run.smithi149.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 369 kB/s | 305 kB 00:00 2024-05-03T17:42:21.702 INFO:teuthology.orchestra.run.smithi149.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 89 kB/s | 58 kB 00:00 2024-05-03T17:42:21.746 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T17:42:21.812 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T17:42:21.912 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T17:42:21.942 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T17:42:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T17:42:22.019 INFO:teuthology.orchestra.run.smithi149.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 135 kB/s | 94 kB 00:00 2024-05-03T17:42:22.185 INFO:teuthology.orchestra.run.smithi149.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-05-03T17:42:22.219 INFO:teuthology.orchestra.run.smithi149.stdout:(102/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 92 kB/s | 64 kB 00:00 2024-05-03T17:42:22.336 INFO:teuthology.orchestra.run.smithi149.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-05-03T17:42:22.386 INFO:teuthology.orchestra.run.smithi149.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-05-03T17:42:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T17:42:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T17:42:22.562 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T17:42:22.578 INFO:teuthology.orchestra.run.smithi149.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-05-03T17:42:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T17:42:22.603 INFO:teuthology.orchestra.run.smithi149.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 101 kB/s | 19 kB 00:00 2024-05-03T17:42:22.662 INFO:teuthology.orchestra.run.smithi149.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-05-03T17:42:22.695 INFO:teuthology.orchestra.run.smithi149.stdout:(108/152): gperftools-libs-2.9.1-2.el9.x86_64.r 863 kB/s | 309 kB 00:00 2024-05-03T17:42:22.721 INFO:teuthology.orchestra.run.smithi149.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 828 kB/s | 49 kB 00:00 2024-05-03T17:42:22.754 INFO:teuthology.orchestra.run.smithi149.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-03T17:42:22.821 INFO:teuthology.orchestra.run.smithi149.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-05-03T17:42:22.922 INFO:teuthology.orchestra.run.smithi149.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-05-03T17:42:22.955 INFO:teuthology.orchestra.run.smithi149.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-05-03T17:42:22.980 INFO:teuthology.orchestra.run.smithi149.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-05-03T17:42:23.014 INFO:teuthology.orchestra.run.smithi149.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-05-03T17:42:23.065 INFO:teuthology.orchestra.run.smithi149.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-05-03T17:42:23.090 INFO:teuthology.orchestra.run.smithi149.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-03T17:42:23.157 INFO:teuthology.orchestra.run.smithi149.stdout:(118/152): libarrow-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 4.4 MB 00:00 2024-05-03T17:42:23.182 INFO:teuthology.orchestra.run.smithi149.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-03T17:42:23.215 INFO:teuthology.orchestra.run.smithi149.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-05-03T17:42:23.250 INFO:teuthology.orchestra.run.smithi149.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-05-03T17:42:23.275 INFO:teuthology.orchestra.run.smithi149.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-03T17:42:23.308 INFO:teuthology.orchestra.run.smithi149.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-03T17:42:23.326 INFO:teuthology.orchestra.run.smithi149.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-05-03T17:42:23.359 INFO:teuthology.orchestra.run.smithi149.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-05-03T17:42:23.376 INFO:teuthology.orchestra.run.smithi149.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-05-03T17:42:23.409 INFO:teuthology.orchestra.run.smithi149.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-05-03T17:42:23.435 INFO:teuthology.orchestra.run.smithi149.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 334 kB/s | 20 kB 00:00 2024-05-03T17:42:23.485 INFO:teuthology.orchestra.run.smithi149.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-05-03T17:42:23.510 INFO:teuthology.orchestra.run.smithi149.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 614 kB/s | 46 kB 00:00 2024-05-03T17:42:23.602 INFO:teuthology.orchestra.run.smithi149.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 674 kB/s | 79 kB 00:00 2024-05-03T17:42:23.627 INFO:teuthology.orchestra.run.smithi149.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 494 kB/s | 58 kB 00:00 2024-05-03T17:42:23.694 INFO:teuthology.orchestra.run.smithi149.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-03T17:42:23.719 INFO:teuthology.orchestra.run.smithi149.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-05-03T17:42:23.753 INFO:teuthology.orchestra.run.smithi149.stdout:(135/152): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-05-03T17:42:23.779 INFO:teuthology.orchestra.run.smithi149.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-05-03T17:42:23.812 INFO:teuthology.orchestra.run.smithi149.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-05-03T17:42:23.846 INFO:teuthology.orchestra.run.smithi149.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-05-03T17:42:23.879 INFO:teuthology.orchestra.run.smithi149.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-03T17:42:23.905 INFO:teuthology.orchestra.run.smithi149.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 640 kB/s | 59 kB 00:00 2024-05-03T17:42:23.930 INFO:teuthology.orchestra.run.smithi149.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-05-03T17:42:23.955 INFO:teuthology.orchestra.run.smithi149.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-03T17:42:23.980 INFO:teuthology.orchestra.run.smithi149.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-03T17:42:24.005 INFO:teuthology.orchestra.run.smithi149.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-03T17:42:24.039 INFO:teuthology.orchestra.run.smithi149.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-03T17:42:24.065 INFO:teuthology.orchestra.run.smithi149.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-03T17:42:24.090 INFO:teuthology.orchestra.run.smithi149.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-03T17:42:24.257 INFO:teuthology.orchestra.run.smithi149.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-05-03T17:42:24.365 INFO:teuthology.orchestra.run.smithi149.stdout:(149/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 11 MB/s | 3.1 MB 00:00 2024-05-03T17:42:24.524 INFO:teuthology.orchestra.run.smithi149.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 5.1 MB/s | 1.3 MB 00:00 2024-05-03T17:42:24.725 INFO:teuthology.orchestra.run.smithi149.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.7 MB/s | 2.4 MB 00:00 2024-05-03T17:42:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T17:42:24.865 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T17:42:25.125 INFO:teuthology.orchestra.run.smithi149.stdout:(152/152): librados2-19.0.0-3450.g9a047fe7.el9. 3.2 MB/s | 3.4 MB 00:01 2024-05-03T17:42:25.133 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-03T17:42:25.133 INFO:teuthology.orchestra.run.smithi149.stdout:Total 8.7 MB/s | 222 MB 00:25 2024-05-03T17:42:25.406 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T17:42:25.444 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T17:42:25.536 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T17:42:25.536 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T17:42:25.537 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T17:42:25.537 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:42:25.570 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T17:42:26.190 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-03T17:42:26.338 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-03T17:42:26.339 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-03T17:42:28.178 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-03T17:42:28.179 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-05-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-05-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-05-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-05-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-05-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-05-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-05-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:31.431 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T17:42:31.447 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T17:42:31.447 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:31.447 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T17:42:31.447 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:32.918 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T17:42:32.944 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T17:42:32.944 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:32.944 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T17:42:32.944 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T17:42:32.944 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T17:42:32.944 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T17:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T17:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T17:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T17:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T17:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:33.636 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T17:42:33.658 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T17:42:33.659 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:33.659 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T17:42:33.659 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T17:42:33.659 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T17:42:33.659 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:33.766 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T17:42:33.823 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T17:42:33.887 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T17:42:34.072 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T17:42:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T17:42:34.207 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T17:42:34.303 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T17:42:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-03T17:42:34.429 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T17:42:34.443 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T17:42:34.468 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T17:42:34.478 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T17:42:34.518 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T17:42:34.521 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T17:42:34.546 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T17:42:34.564 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T17:42:34.589 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T17:42:34.632 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T17:42:34.962 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T17:42:35.006 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T17:42:35.036 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T17:42:35.064 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T17:42:35.067 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T17:42:35.130 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T17:42:35.159 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T17:42:35.187 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T17:42:35.216 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T17:42:35.307 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T17:42:35.336 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T17:42:35.496 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T17:42:35.542 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T17:42:35.569 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T17:42:35.673 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T17:42:35.695 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T17:42:35.758 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T17:42:35.772 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T17:42:35.792 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T17:42:35.988 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T17:42:36.021 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T17:42:36.138 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T17:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T17:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T17:42:36.245 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T17:42:36.279 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T17:42:36.325 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T17:42:36.423 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T17:42:36.463 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T17:42:36.506 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T17:42:36.539 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T17:42:36.567 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T17:42:36.638 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T17:42:36.691 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T17:42:36.720 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T17:42:36.755 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T17:42:36.834 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T17:42:36.866 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T17:42:36.909 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T17:42:36.938 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T17:42:37.010 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T17:42:37.053 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T17:42:37.130 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T17:42:37.190 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T17:42:37.211 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T17:42:37.285 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T17:42:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T17:42:37.334 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T17:42:37.334 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-05-03T17:42:37.335 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-05-03T17:42:37.335 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-05-03T17:42:37.335 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-05-03T17:42:37.335 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-05-03T17:42:37.335 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-05-03T17:42:37.335 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-05-03T17:42:37.335 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:42:37.348 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T17:42:37.490 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T17:42:37.503 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T17:42:37.519 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T17:42:37.519 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:37.519 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T17:42:37.519 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:42:37.522 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T17:42:37.544 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T17:42:37.544 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:37.545 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T17:42:37.545 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T17:42:37.545 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T17:42:37.545 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:37.589 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T17:42:37.600 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T17:42:37.699 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T17:42:37.746 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T17:42:37.899 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T17:42:37.987 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T17:42:38.022 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T17:42:38.064 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T17:42:38.103 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T17:42:38.144 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T17:42:38.300 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T17:42:38.363 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T17:42:38.420 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T17:42:38.494 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T17:42:38.531 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T17:42:38.568 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T17:42:38.624 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T17:42:38.658 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T17:42:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T17:42:38.702 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T17:42:38.708 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T17:42:38.730 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T17:42:38.730 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:38.730 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T17:42:38.731 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T17:42:38.731 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T17:42:38.731 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:38.744 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T17:42:38.775 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T17:42:38.797 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T17:42:38.797 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:38.797 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T17:42:38.797 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:38.851 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T17:42:38.874 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T17:42:38.874 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:38.874 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T17:42:38.874 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T17:42:38.874 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T17:42:38.874 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:38.888 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T17:42:38.949 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T17:42:38.979 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T17:42:39.015 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T17:42:39.021 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T17:42:39.040 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T17:42:39.040 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:39.040 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T17:42:39.041 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T17:42:39.041 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T17:42:39.041 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:42:39.119 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T17:42:39.154 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T17:42:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T17:42:39.194 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T17:42:39.207 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T17:42:39.207 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:39.207 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T17:42:39.208 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T17:42:39.208 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T17:42:39.208 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:39.220 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T17:42:39.245 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T17:42:39.258 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T17:42:39.280 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T17:42:39.280 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:39.280 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T17:42:39.280 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T17:42:39.280 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T17:42:39.280 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:42:39.323 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T17:42:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T17:42:39.660 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T17:42:39.688 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T17:42:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T17:42:39.725 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T17:42:39.748 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T17:42:39.748 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:39.748 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T17:42:39.748 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T17:42:39.748 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T17:42:39.748 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:42:39.762 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T17:42:39.810 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T17:42:39.840 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T17:42:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T17:42:39.874 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T17:42:39.908 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T17:42:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T17:42:39.949 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T17:42:39.968 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T17:42:39.983 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T17:42:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T17:42:40.115 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T17:42:40.145 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T17:42:40.153 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T17:42:40.193 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T17:42:40.201 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T17:42:40.279 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T17:42:40.303 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T17:42:40.356 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T17:42:40.368 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T17:42:40.507 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T17:42:40.552 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T17:42:40.584 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T17:42:40.611 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T17:42:41.056 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T17:42:41.077 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T17:42:41.109 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T17:42:41.136 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T17:42:41.171 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T17:42:41.185 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T17:42:41.200 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T17:42:41.219 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T17:42:41.322 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T17:42:41.378 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T17:42:41.467 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T17:42:41.513 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T17:42:41.694 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T17:42:41.719 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T17:42:41.797 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T17:42:42.239 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T17:42:42.285 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T17:42:42.478 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T17:42:42.595 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T17:42:43.246 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T17:42:43.278 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T17:42:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T17:42:43.547 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T17:42:43.580 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T17:42:43.602 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T17:42:43.603 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:43.603 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T17:42:43.603 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T17:42:43.603 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T17:42:43.603 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:42:43.647 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T17:42:43.658 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T17:42:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T17:42:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T17:42:43.805 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T17:42:43.846 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T17:42:43.888 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T17:42:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T17:42:43.919 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T17:42:43.947 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T17:42:43.981 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T17:42:44.008 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T17:42:44.041 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T17:42:44.133 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T17:42:44.176 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T17:42:44.206 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T17:42:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T17:42:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T17:42:44.487 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T17:42:44.517 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T17:42:44.554 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T17:42:44.731 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T17:42:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T17:42:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T17:42:44.781 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:44.781 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T17:42:44.781 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T17:42:44.781 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T17:42:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:42:44.829 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T17:42:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T17:42:44.856 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T17:42:44.857 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:44.857 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T17:42:44.857 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:42:44.870 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T17:42:44.891 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T17:42:44.901 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T17:42:44.923 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T17:42:44.923 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:44.923 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T17:42:44.923 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T17:42:44.924 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T17:42:44.924 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:42:45.034 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T17:42:45.066 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T17:42:45.145 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T17:42:45.167 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T17:42:45.227 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T17:42:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T17:42:45.249 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:42:45.249 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T17:42:45.249 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T17:42:45.250 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T17:42:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:42:45.720 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T17:42:45.800 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T17:42:45.841 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T17:42:45.924 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T17:42:45.954 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T17:42:45.954 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T17:42:46.022 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T17:42:46.054 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T17:42:46.054 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T17:42:46.124 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T17:42:46.193 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T17:42:47.480 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T17:42:47.522 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T17:42:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T17:42:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T17:42:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T17:42:48.206 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T17:42:48.207 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T17:42:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:42:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T17:42:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T17:42:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T17:42:49.955 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T17:42:49.988 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T17:42:50.015 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T17:42:50.201 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T17:42:50.245 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T17:42:50.274 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T17:42:50.898 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T17:42:50.939 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T17:42:50.959 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T17:42:51.788 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T17:42:51.879 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T17:42:51.919 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T17:42:52.000 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T17:42:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T17:42:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T17:42:52.142 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T17:42:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T17:42:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T17:42:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T17:42:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T17:42:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T17:42:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T17:42:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T17:42:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T17:42:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T17:42:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T17:42:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T17:42:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T17:42:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T17:42:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T17:42:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T17:42:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T17:42:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T17:42:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T17:42:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T17:42:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T17:42:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T17:42:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T17:42:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T17:42:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T17:42:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T17:42:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T17:42:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T17:42:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T17:42:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T17:42:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T17:42:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T17:42:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T17:42:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T17:42:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T17:42:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T17:42:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T17:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T17:42:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T17:42:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T17:42:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T17:42:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T17:42:52.181 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T17:42:52.181 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T17:42:52.256 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T17:42:52.322 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T17:42:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T17:42:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:54.270 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T17:42:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T17:42:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T17:42:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T17:42:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T17:42:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T17:42:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T17:42:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T17:42:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T17:42:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T17:42:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T17:42:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T17:42:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T17:42:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T17:42:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T17:42:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:42:54.282 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-05-03T17:42:54.688 DEBUG:teuthology.parallel:result is None 2024-05-03T17:42:58.003 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T17:42:58.003 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T17:42:58.003 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T17:42:58.003 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T17:42:58.003 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T17:42:58.003 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T17:42:58.003 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T17:42:58.003 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T17:42:58.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T17:42:58.005 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T17:42:58.005 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T17:42:58.005 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T17:42:58.005 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T17:42:58.005 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T17:42:58.005 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T17:42:58.005 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T17:42:58.005 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T17:42:58.018 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T17:42:58.018 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T17:42:58.018 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T17:42:58.019 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T17:42:58.020 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T17:42:58.021 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T17:42:58.022 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T17:42:58.023 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T17:42:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T17:42:58.025 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T17:42:58.026 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T17:42:58.027 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T17:42:58.027 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T17:42:58.027 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T17:42:58.027 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T17:42:58.027 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T17:42:58.027 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T17:42:58.027 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T17:42:58.027 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T17:42:58.027 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T17:42:58.027 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T17:42:58.027 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T17:42:59.823 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T17:42:59.823 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-05-03T17:42:59.823 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-05-03T17:42:59.823 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-05-03T17:42:59.823 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-05-03T17:42:59.823 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-05-03T17:42:59.824 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-05-03T17:42:59.824 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-05-03T17:42:59.824 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:42:59.991 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T17:43:00.007 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T17:43:00.007 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:43:00.007 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T17:43:00.007 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T17:43:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:00.102 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.103 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.104 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T17:43:00.105 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T17:43:00.106 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T17:43:00.107 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T17:43:00.108 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T17:43:00.109 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T17:43:00.110 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.111 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T17:43:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T17:43:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T17:43:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T17:43:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T17:43:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T17:43:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T17:43:00.112 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:43:00.113 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-03T17:43:00.468 DEBUG:teuthology.parallel:result is None 2024-05-03T17:43:01.512 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T17:43:01.538 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T17:43:01.538 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:43:01.538 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T17:43:01.538 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T17:43:01.538 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T17:43:01.539 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:43:01.762 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T17:43:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T17:43:01.785 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:43:01.785 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T17:43:01.785 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T17:43:01.785 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T17:43:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:43:02.221 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T17:43:02.243 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T17:43:02.243 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:43:02.243 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T17:43:02.243 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T17:43:02.243 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T17:43:02.243 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:43:02.360 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T17:43:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T17:43:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T17:43:02.678 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T17:43:02.731 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T17:43:02.811 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T17:43:02.902 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T17:43:03.043 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T17:43:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T17:43:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T17:43:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T17:43:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T17:43:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T17:43:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T17:43:03.734 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T17:43:03.761 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T17:43:04.839 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T17:43:04.880 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T17:43:05.854 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T17:43:05.888 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T17:43:06.006 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T17:43:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T17:43:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T17:43:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T17:43:06.222 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:43:06.222 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T17:43:06.222 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T17:43:06.222 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T17:43:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:43:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T17:43:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T17:43:06.384 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T17:43:06.432 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T17:43:07.358 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T17:43:07.393 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T17:43:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T17:43:07.416 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:43:07.416 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T17:43:07.416 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T17:43:07.416 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T17:43:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:43:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T17:43:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T17:43:07.483 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:43:07.483 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T17:43:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:43:07.528 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T17:43:07.550 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T17:43:07.551 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:43:07.551 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T17:43:07.551 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T17:43:07.551 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T17:43:07.551 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:43:07.862 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T17:43:07.885 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T17:43:07.885 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T17:43:07.885 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T17:43:07.885 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T17:43:07.885 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T17:43:07.885 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:43:12.549 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T17:43:12.591 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T17:43:12.632 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T17:43:12.663 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T17:43:12.691 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T17:43:12.878 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T17:43:12.920 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T17:43:12.949 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T17:43:13.591 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T17:43:13.630 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T17:43:13.650 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T17:43:14.541 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T17:43:14.626 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T17:43:14.669 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T17:43:14.763 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T17:43:14.805 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T17:43:14.805 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T17:43:14.889 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T17:43:14.922 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T17:43:14.922 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T17:43:15.016 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T17:43:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T17:43:21.013 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T17:43:21.013 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T17:43:21.013 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T17:43:21.013 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T17:43:21.013 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T17:43:21.013 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T17:43:21.013 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T17:43:21.013 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T17:43:21.013 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T17:43:21.014 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T17:43:21.015 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T17:43:21.015 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T17:43:21.015 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T17:43:21.015 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T17:43:21.015 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T17:43:21.015 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T17:43:21.015 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T17:43:21.015 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T17:43:21.027 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T17:43:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T17:43:21.029 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T17:43:21.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T17:43:21.031 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T17:43:21.032 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T17:43:21.033 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T17:43:21.034 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T17:43:21.035 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T17:43:21.036 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T17:43:21.036 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T17:43:21.036 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T17:43:21.036 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T17:43:23.087 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T17:43:23.087 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:43:23.087 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-05-03T17:43:23.087 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.087 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.088 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.089 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.090 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T17:43:23.091 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T17:43:23.092 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T17:43:23.093 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T17:43:23.094 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T17:43:23.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T17:43:23.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.097 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.098 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T17:43:23.098 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T17:43:23.098 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T17:43:23.098 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T17:43:23.098 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T17:43:23.098 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T17:43:23.098 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T17:43:23.098 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T17:43:23.098 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T17:43:23.099 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-03T17:43:23.472 DEBUG:teuthology.parallel:result is None 2024-05-03T17:43:23.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:43:23.613 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T17:43:23.640 INFO:teuthology.orchestra.run.smithi122.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T17:43:23.640 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T17:43:23.640 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T17:43:23.642 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:43:23.858 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T17:43:23.881 INFO:teuthology.orchestra.run.smithi149.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T17:43:23.881 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T17:43:23.881 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T17:43:23.883 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:43:24.099 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T17:43:24.126 INFO:teuthology.orchestra.run.smithi192.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T17:43:24.126 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T17:43:24.126 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T17:43:24.128 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-03T17:43:24.128 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:43:24.128 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T17:43:24.159 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T17:43:24.159 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T17:43:24.186 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-03T17:43:24.186 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T17:43:24.217 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-03T17:43:24.217 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:43:24.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T17:43:24.244 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T17:43:24.309 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T17:43:24.310 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T17:43:24.333 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T17:43:24.398 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-03T17:43:24.398 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T17:43:24.426 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T17:43:24.495 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-03T17:43:24.495 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:43:24.495 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T17:43:24.522 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T17:43:24.586 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T17:43:24.586 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T17:43:24.609 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T17:43:24.673 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-03T17:43:24.673 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T17:43:24.701 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T17:43:24.769 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-03T17:43:24.769 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:43:24.769 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T17:43:24.793 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T17:43:24.856 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T17:43:24.856 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T17:43:24.881 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T17:43:24.944 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-03T17:43:24.944 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T17:43:24.972 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T17:43:25.039 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-03T17:43:25.141 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'subvols': {'create': 2}} 2024-05-03T17:43:25.141 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:43:25.142 INFO:tasks.cephadm:Cluster fsid is a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:43:25.142 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-03T17:43:25.142 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.122', 'mon.b': '172.21.15.149', 'mon.c': '172.21.15.192'} 2024-05-03T17:43:25.142 INFO:tasks.cephadm:First mon is mon.a on smithi122 2024-05-03T17:43:25.142 INFO:tasks.cephadm:First mgr is x 2024-05-03T17:43:25.142 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-03T17:43:25.143 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname $(hostname -s) 2024-05-03T17:43:25.167 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-05-03T17:43:25.191 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2024-05-03T17:43:25.219 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-03T17:43:25.219 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:43:25.359 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79034/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3450-g9a047fe7', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3450.g9a047fe7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-02 00:36:37.664792', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240501.193033', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-03T17:43:25.498 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-rishabh-testing-20240501.193033, sha1 9a047fe7599a549042d06a3fb5b6813a203eabea from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:43:25.500 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm 2024-05-03T17:43:25.500 DEBUG:teuthology.orchestra.run.smithi122:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T17:43:25.940 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 17:43 /home/ubuntu/cephtest/cephadm 2024-05-03T17:43:25.940 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T17:43:26.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 17:43 /home/ubuntu/cephtest/cephadm 2024-05-03T17:43:26.401 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T17:43:26.967 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 17:43 /home/ubuntu/cephtest/cephadm 2024-05-03T17:43:26.968 DEBUG:teuthology.orchestra.run.smithi122:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T17:43:26.986 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T17:43:27.002 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T17:43:27.038 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea on all hosts... 2024-05-03T17:43:27.038 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T17:43:27.040 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T17:43:27.045 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T17:43:28.553 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T17:43:28.645 INFO:teuthology.orchestra.run.smithi122.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T17:43:28.732 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T17:43:56.812 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-03T17:43:56.812 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T17:43:56.813 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T17:43:56.813 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-05-03T17:43:56.813 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T17:43:56.813 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-03T17:43:56.813 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-03T17:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-03T17:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T17:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T17:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2024-05-03T17:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T17:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-03T17:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-03T17:44:04.126 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-05-03T17:44:04.127 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T17:44:04.127 INFO:teuthology.orchestra.run.smithi122.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T17:44:04.127 INFO:teuthology.orchestra.run.smithi122.stdout: "repo_digests": [ 2024-05-03T17:44:04.127 INFO:teuthology.orchestra.run.smithi122.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T17:44:04.127 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-05-03T17:44:04.127 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-05-03T17:44:04.150 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph 2024-05-03T17:44:04.190 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-05-03T17:44:04.221 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2024-05-03T17:44:04.255 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /etc/ceph 2024-05-03T17:44:04.278 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-05-03T17:44:04.305 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2024-05-03T17:44:04.333 INFO:tasks.cephadm:Writing seed config... 2024-05-03T17:44:04.334 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-03T17:44:04.334 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-03T17:44:04.334 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-03T17:44:04.334 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-03T17:44:04.335 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-03T17:44:04.336 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-03T17:44:04.337 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-03T17:44:04.337 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-03T17:44:04.337 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-03T17:44:04.337 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-03T17:44:04.337 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-03T17:44:04.337 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-03T17:44:04.338 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-03T17:44:04.338 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-03T17:44:04.338 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-03T17:44:04.338 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-03T17:44:04.338 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-03T17:44:04.338 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-03T17:44:04.339 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-03T17:44:04.339 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-03T17:44:04.339 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-03T17:44:04.339 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-03T17:44:04.340 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:44:04.340 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-03T17:44:04.356 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a46fc846-0974-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-03T17:44:04.356 DEBUG:teuthology.orchestra.run.smithi122:mon.a> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.a.service 2024-05-03T17:44:04.398 DEBUG:teuthology.orchestra.run.smithi122:mgr.x> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.x.service 2024-05-03T17:44:04.441 INFO:tasks.cephadm:Bootstrapping... 2024-05-03T17:44:04.442 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea -v bootstrap --fsid a46fc846-0974-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.122 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-03T17:44:04.631 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-05-03T17:44:04.631 INFO:teuthology.orchestra.run.smithi122.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea', '-v', 'bootstrap', '--fsid', 'a46fc846-0974-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.122', '--skip-admin-label'] 2024-05-03T17:44:04.631 INFO:teuthology.orchestra.run.smithi122.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-03T17:44:04.631 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying podman|docker is present... 2024-05-03T17:44:04.665 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-03T17:44:04.665 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying lvm2 is present... 2024-05-03T17:44:04.665 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying time synchronization is in place... 2024-05-03T17:44:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-03T17:44:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-03T17:44:04.678 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-03T17:44:04.678 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-05-03T17:44:04.684 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout enabled 2024-05-03T17:44:04.690 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout active 2024-05-03T17:44:04.691 INFO:teuthology.orchestra.run.smithi122.stdout:Unit chronyd.service is enabled and running 2024-05-03T17:44:04.691 INFO:teuthology.orchestra.run.smithi122.stdout:Repeating the final host check... 2024-05-03T17:44:04.723 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-03T17:44:04.723 INFO:teuthology.orchestra.run.smithi122.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-03T17:44:04.723 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl is present 2024-05-03T17:44:04.723 INFO:teuthology.orchestra.run.smithi122.stdout:lvcreate is present 2024-05-03T17:44:04.729 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-03T17:44:04.729 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-03T17:44:04.734 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-03T17:44:04.735 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-05-03T17:44:04.741 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout enabled 2024-05-03T17:44:04.747 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout active 2024-05-03T17:44:04.747 INFO:teuthology.orchestra.run.smithi122.stdout:Unit chronyd.service is enabled and running 2024-05-03T17:44:04.747 INFO:teuthology.orchestra.run.smithi122.stdout:Host looks OK 2024-05-03T17:44:04.748 INFO:teuthology.orchestra.run.smithi122.stdout:Cluster fsid: a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:04.748 INFO:teuthology.orchestra.run.smithi122.stdout:Acquiring lock 140407893204272 on /run/cephadm/a46fc846-0974-11ef-bc97-c7b262605968.lock 2024-05-03T17:44:04.748 INFO:teuthology.orchestra.run.smithi122.stdout:Lock 140407893204272 acquired on /run/cephadm/a46fc846-0974-11ef-bc97-c7b262605968.lock 2024-05-03T17:44:04.748 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying IP 172.21.15.122 port 3300 ... 2024-05-03T17:44:04.748 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying IP 172.21.15.122 port 6789 ... 2024-05-03T17:44:04.748 INFO:teuthology.orchestra.run.smithi122.stdout:Base mon IP(s) is [172.21.15.122:3300, 172.21.15.122:6789], mon addrv is [v2:172.21.15.122:3300,v1:172.21.15.122:6789] 2024-05-03T17:44:04.751 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.122 metric 100 2024-05-03T17:44:04.751 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.122 metric 100 2024-05-03T17:44:04.754 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-03T17:44:04.754 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-05-03T17:44:04.754 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-05-03T17:44:04.756 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-03T17:44:04.756 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-03T17:44:04.756 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-03T17:44:04.756 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-03T17:44:04.756 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout inet6 fe80::1da6:cc88:1934:4e50/64 scope link noprefixroute 2024-05-03T17:44:04.757 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-03T17:44:04.757 INFO:teuthology.orchestra.run.smithi122.stdout:Mon IP `172.21.15.122` is in CIDR network `172.21.0.0/20` 2024-05-03T17:44:04.758 INFO:teuthology.orchestra.run.smithi122.stdout:Mon IP `172.21.15.122` is in CIDR network `172.21.0.0/20` 2024-05-03T17:44:04.758 INFO:teuthology.orchestra.run.smithi122.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-03T17:44:04.758 INFO:teuthology.orchestra.run.smithi122.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-03T17:44:04.758 INFO:teuthology.orchestra.run.smithi122.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T17:44:05.254 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 2024-05-03T17:44:05.254 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T17:44:05.254 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Getting image source signatures 2024-05-03T17:44:05.254 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying blob sha256:8173fd91c96404ae0424bb08037c57749051dcbdfaeee9354a65e201dad7b0e2 2024-05-03T17:44:05.254 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying blob sha256:e2f2fd2adca92139da4b4b907fa90a6775d3eb4eddaa977c42ef37c66eea70bd 2024-05-03T17:44:05.254 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying config sha256:ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 2024-05-03T17:44:05.254 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-03T17:44:06.195 INFO:teuthology.orchestra.run.smithi122.stdout:ceph: stdout ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev) 2024-05-03T17:44:06.195 INFO:teuthology.orchestra.run.smithi122.stdout:Ceph version: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev) 2024-05-03T17:44:06.195 INFO:teuthology.orchestra.run.smithi122.stdout:Extracting ceph user uid/gid from container image... 2024-05-03T17:44:07.159 INFO:teuthology.orchestra.run.smithi122.stdout:stat: stdout 167 167 2024-05-03T17:44:07.159 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial keys... 2024-05-03T17:44:08.176 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQBnIjVm/I/JHBAA4+HupNdPwK8Qkfz+w225NQ== 2024-05-03T17:44:09.081 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQBoIjVmqZMpHRAAFIJYwt5Ahjnzjea3qkrAHA== 2024-05-03T17:44:10.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQBpIjVmSMEPGBAAPPDEOuEge3gLBpedU/g7xg== 2024-05-03T17:44:10.004 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial monmap... 2024-05-03T17:44:11.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-03T17:44:11.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-03T17:44:11.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:11.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-03T17:44:11.002 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool for a [v2:172.21.15.122:3300,v1:172.21.15.122:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-03T17:44:11.002 INFO:teuthology.orchestra.run.smithi122.stdout:setting min_mon_release = quincy 2024-05-03T17:44:11.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: set fsid to a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:11.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-03T17:44:11.003 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:44:11.003 INFO:teuthology.orchestra.run.smithi122.stdout:Creating mon... 2024-05-03T17:44:11.790 INFO:teuthology.orchestra.run.smithi122.stdout:create mon.a on 2024-05-03T17:44:12.043 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-03T17:44:12.320 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-03T17:44:12.860 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a46fc846-0974-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-a46fc846-0974-11ef-bc97-c7b262605968.target. 2024-05-03T17:44:12.861 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a46fc846-0974-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-a46fc846-0974-11ef-bc97-c7b262605968.target. 2024-05-03T17:44:13.758 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.a 2024-05-03T17:44:13.758 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-03T17:44:14.091 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a46fc846-0974-11ef-bc97-c7b262605968.target.wants/ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a46fc846-0974-11ef-bc97-c7b262605968@.service. 2024-05-03T17:44:14.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:14 smithi122 podman[29353]: 2024-05-03 17:44:14.243299683 +0000 UTC m=+0.017466085 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:44:15.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:15 smithi122 podman[29353]: 2024-05-03 17:44:15.549750805 +0000 UTC m=+1.323917199 container create ddaf75b5e79418168eb8e390167a8a0fea4e520353da94b6741d2fd1657f941f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430) 2024-05-03T17:44:17.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:17 smithi122 podman[29353]: 2024-05-03 17:44:17.365429985 +0000 UTC m=+3.139596380 container init ddaf75b5e79418168eb8e390167a8a0fea4e520353da94b6741d2fd1657f941f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T17:44:17.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:17 smithi122 podman[29353]: 2024-05-03 17:44:17.369193194 +0000 UTC m=+3.143359590 container start ddaf75b5e79418168eb8e390167a8a0fea4e520353da94b6741d2fd1657f941f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T17:44:17.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:17 smithi122 ceph-mon[29372]: mkfs a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:17.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:17 smithi122 ceph-mon[29372]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T17:44:18.161 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T17:44:18.161 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-05-03T17:44:18.169 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T17:44:18.169 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-05-03T17:44:18.169 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-05-03T17:44:18.169 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to enable service . firewalld.service is not available 2024-05-03T17:44:18.170 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mon to start... 2024-05-03T17:44:18.170 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mon... 2024-05-03T17:44:18.417 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 bash[29353]: ddaf75b5e79418168eb8e390167a8a0fea4e520353da94b6741d2fd1657f941f 2024-05-03T17:44:18.418 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 systemd[1]: Started Ceph mon.a for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:44:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 ceph-mon[29372]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T17:44:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 ceph-mon[29372]: monmap epoch 1 2024-05-03T17:44:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 ceph-mon[29372]: fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 ceph-mon[29372]: last_changed 2024-05-03T17:44:10.382958+0000 2024-05-03T17:44:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 ceph-mon[29372]: created 2024-05-03T17:44:10.382958+0000 2024-05-03T17:44:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 ceph-mon[29372]: min_mon_release 19 (squid) 2024-05-03T17:44:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 ceph-mon[29372]: election_strategy: 1 2024-05-03T17:44:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 ceph-mon[29372]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-05-03T17:44:18.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 ceph-mon[29372]: fsmap 2024-05-03T17:44:18.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 ceph-mon[29372]: osdmap e1: 0 total, 0 up, 0 in 2024-05-03T17:44:18.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:18 smithi122 ceph-mon[29372]: mgrmap e1: no daemons active 2024-05-03T17:44:21.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:21 smithi122 ceph-mon[29372]: from='client.? 172.21.15.122:0/3644352037' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-03T17:44:25.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout cluster: 2024-05-03T17:44:25.036 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout id: a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:25.036 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-03T17:44:25.036 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:25.036 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout services: 2024-05-03T17:44:25.036 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 3s) 2024-05-03T17:44:25.036 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-03T17:44:25.036 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-03T17:44:25.036 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:25.036 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout data: 2024-05-03T17:44:25.036 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-03T17:44:25.036 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout pgs: 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.274+0000 7f63296d2700 1 Processor -- start 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.274+0000 7f63296d2700 1 -- start start 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.275+0000 7f63296d2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f6324108170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.275+0000 7f63296d2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6324108740 con 0x7f6324107d50 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.275+0000 7f6322ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f6324108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.275+0000 7f6322ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f6324108170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33476/0 (socket says 172.21.15.122:33476) 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.275+0000 7f6322ffd700 1 -- 172.21.15.122:0/780811045 learned_addr learned my addr 172.21.15.122:0/780811045 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.275+0000 7f6322ffd700 1 -- 172.21.15.122:0/780811045 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6324108f50 con 0x7f6324107d50 2024-05-03T17:44:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.276+0000 7f6322ffd700 1 --2- 172.21.15.122:0/780811045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f6324108170 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f63140098a0 tx=0x7f6314009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7894e3dd0be27f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.276+0000 7f6321ffb700 1 -- 172.21.15.122:0/780811045 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6314004160 con 0x7f6324107d50 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.276+0000 7f6321ffb700 1 -- 172.21.15.122:0/780811045 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6314004300 con 0x7f6324107d50 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.276+0000 7f6321ffb700 1 -- 172.21.15.122:0/780811045 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63140036a0 con 0x7f6324107d50 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.276+0000 7f63296d2700 1 -- 172.21.15.122:0/780811045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 msgr2=0x7f6324108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.276+0000 7f63296d2700 1 --2- 172.21.15.122:0/780811045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f6324108170 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f63140098a0 tx=0x7f6314009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.276+0000 7f63296d2700 1 -- 172.21.15.122:0/780811045 shutdown_connections 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.276+0000 7f63296d2700 1 --2- 172.21.15.122:0/780811045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f6324108170 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.276+0000 7f63296d2700 1 -- 172.21.15.122:0/780811045 >> 172.21.15.122:0/780811045 conn(0x7f63241032f0 msgr2=0x7f6324105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.276+0000 7f63296d2700 1 -- 172.21.15.122:0/780811045 shutdown_connections 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.276+0000 7f63296d2700 1 -- 172.21.15.122:0/780811045 wait complete. 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.277+0000 7f63296d2700 1 Processor -- start 2024-05-03T17:44:25.038 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.277+0000 7f63296d2700 1 -- start start 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.277+0000 7f63296d2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f632407ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.277+0000 7f63296d2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6324109480 con 0x7f6324107d50 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.277+0000 7f6322ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f632407ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.277+0000 7f6322ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f632407ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33488/0 (socket says 172.21.15.122:33488) 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.277+0000 7f6322ffd700 1 -- 172.21.15.122:0/3644352037 learned_addr learned my addr 172.21.15.122:0/3644352037 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.278+0000 7f6322ffd700 1 -- 172.21.15.122:0/3644352037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632419f370 con 0x7f6324107d50 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.278+0000 7f6322ffd700 1 --2- 172.21.15.122:0/3644352037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f632407ca60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6314000c00 tx=0x7f6314004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.278+0000 7f630bfff700 1 -- 172.21.15.122:0/3644352037 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6314003db0 con 0x7f6324107d50 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.278+0000 7f630bfff700 1 -- 172.21.15.122:0/3644352037 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6314011630 con 0x7f6324107d50 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.278+0000 7f63296d2700 1 -- 172.21.15.122:0/3644352037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632419f540 con 0x7f6324107d50 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.278+0000 7f630bfff700 1 -- 172.21.15.122:0/3644352037 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6314011950 con 0x7f6324107d50 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.278+0000 7f63296d2700 1 -- 172.21.15.122:0/3644352037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632419f8e0 con 0x7f6324107d50 2024-05-03T17:44:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.278+0000 7f630bfff700 1 -- 172.21.15.122:0/3644352037 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f631401a460 con 0x7f6324107d50 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.278+0000 7f630bfff700 1 -- 172.21.15.122:0/3644352037 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f631401b780 con 0x7f6324107d50 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.279+0000 7f63296d2700 1 -- 172.21.15.122:0/3644352037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632406a5c0 con 0x7f6324107d50 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.281+0000 7f630bfff700 1 -- 172.21.15.122:0/3644352037 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6314031020 con 0x7f6324107d50 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.353+0000 7f63296d2700 1 -- 172.21.15.122:0/3644352037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f632402e510 con 0x7f6324107d50 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.353+0000 7f630bfff700 1 -- 172.21.15.122:0/3644352037 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7f631401b9d0 con 0x7f6324107d50 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.354+0000 7f63296d2700 1 -- 172.21.15.122:0/3644352037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 msgr2=0x7f632407ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.354+0000 7f63296d2700 1 --2- 172.21.15.122:0/3644352037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f632407ca60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6314000c00 tx=0x7f6314004750 comp rx=0 tx=0).stop 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.354+0000 7f63296d2700 1 -- 172.21.15.122:0/3644352037 shutdown_connections 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.354+0000 7f63296d2700 1 --2- 172.21.15.122:0/3644352037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6324107d50 0x7f632407ca60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.354+0000 7f63296d2700 1 -- 172.21.15.122:0/3644352037 >> 172.21.15.122:0/3644352037 conn(0x7f63241032f0 msgr2=0x7f6324192100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.354+0000 7f63296d2700 1 -- 172.21.15.122:0/3644352037 shutdown_connections 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:21.354+0000 7f63296d2700 1 -- 172.21.15.122:0/3644352037 wait complete. 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:mon is available 2024-05-03T17:44:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:Assimilating anything we can from ceph.conf... 2024-05-03T17:44:25.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:25 smithi122 ceph-mon[29372]: from='client.? 172.21.15.122:0/1928754697' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-03T17:44:25.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:25 smithi122 ceph-mon[29372]: from='client.? 172.21.15.122:0/1928754697' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-03T17:44:26.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:26.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [global] 2024-05-03T17:44:26.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout fsid = a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:26.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-03T17:44:26.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.122:3300,v1:172.21.15.122:6789] 2024-05-03T17:44:26.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-03T17:44:26.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-03T17:44:26.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-03T17:44:26.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-03T17:44:26.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [osd] 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.425+0000 7fe79acc2700 1 Processor -- start 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.425+0000 7fe79acc2700 1 -- start start 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.425+0000 7fe79acc2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe794109230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.425+0000 7fe79acc2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe794109770 con 0x7fe794108e10 2024-05-03T17:44:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.425+0000 7fe798a5e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe794109230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.425+0000 7fe798a5e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe794109230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33500/0 (socket says 172.21.15.122:33500) 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.425+0000 7fe798a5e700 1 -- 172.21.15.122:0/3690054106 learned_addr learned my addr 172.21.15.122:0/3690054106 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.425+0000 7fe798a5e700 1 -- 172.21.15.122:0/3690054106 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe794109f70 con 0x7fe794108e10 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.426+0000 7fe798a5e700 1 --2- 172.21.15.122:0/3690054106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe794109230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe7880098a0 tx=0x7fe788009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf704d93ce2bfa57 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.426+0000 7fe7937fe700 1 -- 172.21.15.122:0/3690054106 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe788004160 con 0x7fe794108e10 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.426+0000 7fe7937fe700 1 -- 172.21.15.122:0/3690054106 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe788004300 con 0x7fe794108e10 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.426+0000 7fe7937fe700 1 -- 172.21.15.122:0/3690054106 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7880036a0 con 0x7fe794108e10 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.426+0000 7fe79acc2700 1 -- 172.21.15.122:0/3690054106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 msgr2=0x7fe794109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.426+0000 7fe79acc2700 1 --2- 172.21.15.122:0/3690054106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe794109230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe7880098a0 tx=0x7fe788009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.426+0000 7fe79acc2700 1 -- 172.21.15.122:0/3690054106 shutdown_connections 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.426+0000 7fe79acc2700 1 --2- 172.21.15.122:0/3690054106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe794109230 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.426+0000 7fe79acc2700 1 -- 172.21.15.122:0/3690054106 >> 172.21.15.122:0/3690054106 conn(0x7fe7941043d0 msgr2=0x7fe7941067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.426+0000 7fe79acc2700 1 -- 172.21.15.122:0/3690054106 shutdown_connections 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.426+0000 7fe79acc2700 1 -- 172.21.15.122:0/3690054106 wait complete. 2024-05-03T17:44:26.052 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.427+0000 7fe79acc2700 1 Processor -- start 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.427+0000 7fe79acc2700 1 -- start start 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.427+0000 7fe79acc2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe79407f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.427+0000 7fe79acc2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410a440 con 0x7fe794108e10 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.427+0000 7fe798a5e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe79407f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.427+0000 7fe798a5e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe79407f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33514/0 (socket says 172.21.15.122:33514) 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.427+0000 7fe798a5e700 1 -- 172.21.15.122:0/1928754697 learned_addr learned my addr 172.21.15.122:0/1928754697 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.427+0000 7fe798a5e700 1 -- 172.21.15.122:0/1928754697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79407f950 con 0x7fe794108e10 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.428+0000 7fe798a5e700 1 --2- 172.21.15.122:0/1928754697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe79407f410 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe788000c00 tx=0x7fe788004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.428+0000 7fe791ffb700 1 -- 172.21.15.122:0/1928754697 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe788003db0 con 0x7fe794108e10 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.428+0000 7fe791ffb700 1 -- 172.21.15.122:0/1928754697 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe788011630 con 0x7fe794108e10 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.428+0000 7fe79acc2700 1 -- 172.21.15.122:0/1928754697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79407db40 con 0x7fe794108e10 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.428+0000 7fe791ffb700 1 -- 172.21.15.122:0/1928754697 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe788011950 con 0x7fe794108e10 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.428+0000 7fe79acc2700 1 -- 172.21.15.122:0/1928754697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79407e080 con 0x7fe794108e10 2024-05-03T17:44:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.428+0000 7fe791ffb700 1 -- 172.21.15.122:0/1928754697 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe788011af0 con 0x7fe794108e10 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.428+0000 7fe791ffb700 1 -- 172.21.15.122:0/1928754697 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe78801c450 con 0x7fe794108e10 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.429+0000 7fe79acc2700 1 -- 172.21.15.122:0/1928754697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79406a5c0 con 0x7fe794108e10 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.431+0000 7fe791ffb700 1 -- 172.21.15.122:0/1928754697 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe788032020 con 0x7fe794108e10 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.499+0000 7fe79acc2700 1 -- 172.21.15.122:0/1928754697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe794195950 con 0x7fe794108e10 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.502+0000 7fe791ffb700 1 -- 172.21.15.122:0/1928754697 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+467 (secure 0 0 0) 0x7fe794195950 con 0x7fe794108e10 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.502+0000 7fe791ffb700 1 -- 172.21.15.122:0/1928754697 <== mon.0 v2:172.21.15.122:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe788004260 con 0x7fe794108e10 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.503+0000 7fe79acc2700 1 -- 172.21.15.122:0/1928754697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 msgr2=0x7fe79407f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.503+0000 7fe79acc2700 1 --2- 172.21.15.122:0/1928754697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe79407f410 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe788000c00 tx=0x7fe788004750 comp rx=0 tx=0).stop 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.503+0000 7fe79acc2700 1 -- 172.21.15.122:0/1928754697 shutdown_connections 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.503+0000 7fe79acc2700 1 --2- 172.21.15.122:0/1928754697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe794108e10 0x7fe79407f410 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.503+0000 7fe79acc2700 1 -- 172.21.15.122:0/1928754697 >> 172.21.15.122:0/1928754697 conn(0x7fe7941043d0 msgr2=0x7fe794193000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.503+0000 7fe79acc2700 1 -- 172.21.15.122:0/1928754697 shutdown_connections 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:25.503+0000 7fe79acc2700 1 -- 172.21.15.122:0/1928754697 wait complete. 2024-05-03T17:44:26.054 INFO:teuthology.orchestra.run.smithi122.stdout:Generating new minimal ceph.conf... 2024-05-03T17:44:26.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:26 smithi122 ceph-mon[29372]: from='client.? 172.21.15.122:0/1107494783' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:44:27.062 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.448+0000 7f4ecc60b700 1 Processor -- start 2024-05-03T17:44:27.062 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.448+0000 7f4ecc60b700 1 -- start start 2024-05-03T17:44:27.062 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.449+0000 7f4ecc60b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4108130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:27.062 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.449+0000 7f4ecc60b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4108700 con 0x7f4ec4107d10 2024-05-03T17:44:27.062 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.449+0000 7f4eca3a7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:27.062 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.449+0000 7f4eca3a7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4108130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33522/0 (socket says 172.21.15.122:33522) 2024-05-03T17:44:27.062 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.449+0000 7f4eca3a7700 1 -- 172.21.15.122:0/1758486891 learned_addr learned my addr 172.21.15.122:0/1758486891 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:27.062 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.449+0000 7f4eca3a7700 1 -- 172.21.15.122:0/1758486891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec4108f10 con 0x7f4ec4107d10 2024-05-03T17:44:27.062 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.449+0000 7f4eca3a7700 1 --2- 172.21.15.122:0/1758486891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4108130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00098a0 tx=0x7f4ec0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291c26c8c9fc7cec server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:27.062 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.450+0000 7f4ec93a5700 1 -- 172.21.15.122:0/1758486891 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ec0004160 con 0x7f4ec4107d10 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.450+0000 7f4ec93a5700 1 -- 172.21.15.122:0/1758486891 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4ec0004300 con 0x7f4ec4107d10 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.450+0000 7f4ec93a5700 1 -- 172.21.15.122:0/1758486891 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ec0011af0 con 0x7f4ec4107d10 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.450+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1758486891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 msgr2=0x7f4ec4108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.450+0000 7f4ecc60b700 1 --2- 172.21.15.122:0/1758486891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4108130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00098a0 tx=0x7f4ec0009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.450+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1758486891 shutdown_connections 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.450+0000 7f4ecc60b700 1 --2- 172.21.15.122:0/1758486891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4108130 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.450+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1758486891 >> 172.21.15.122:0/1758486891 conn(0x7f4ec41032f0 msgr2=0x7f4ec4105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.450+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1758486891 shutdown_connections 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.450+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1758486891 wait complete. 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.451+0000 7f4ecc60b700 1 Processor -- start 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.451+0000 7f4ecc60b700 1 -- start start 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.451+0000 7f4ecc60b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4192e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.451+0000 7f4ecc60b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec410c2c0 con 0x7f4ec4107d10 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.451+0000 7f4eca3a7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4192e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.451+0000 7f4eca3a7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4192e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33528/0 (socket says 172.21.15.122:33528) 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.451+0000 7f4eca3a7700 1 -- 172.21.15.122:0/1107494783 learned_addr learned my addr 172.21.15.122:0/1107494783 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.451+0000 7f4eca3a7700 1 -- 172.21.15.122:0/1107494783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec41933c0 con 0x7f4ec4107d10 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.452+0000 7f4eca3a7700 1 --2- 172.21.15.122:0/1107494783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4192e80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00117b0 tx=0x7f4ec0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.452+0000 7f4ebb7fe700 1 -- 172.21.15.122:0/1107494783 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ec0003af0 con 0x7f4ec4107d10 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.452+0000 7f4ebb7fe700 1 -- 172.21.15.122:0/1107494783 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4ec0020070 con 0x7f4ec4107d10 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.452+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1107494783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec41915b0 con 0x7f4ec4107d10 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.452+0000 7f4ebb7fe700 1 -- 172.21.15.122:0/1107494783 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ec0019630 con 0x7f4ec4107d10 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.452+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1107494783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec4191af0 con 0x7f4ec4107d10 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.452+0000 7f4ebb7fe700 1 -- 172.21.15.122:0/1107494783 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ec0019c30 con 0x7f4ec4107d10 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.452+0000 7f4ebb7fe700 1 -- 172.21.15.122:0/1107494783 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4ec000b720 con 0x7f4ec4107d10 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.453+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1107494783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec406a5c0 con 0x7f4ec4107d10 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.455+0000 7f4ebb7fe700 1 -- 172.21.15.122:0/1107494783 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4ec001a9c0 con 0x7f4ec4107d10 2024-05-03T17:44:27.064 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.522+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1107494783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f4ec4196790 con 0x7f4ec4107d10 2024-05-03T17:44:27.065 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.522+0000 7f4ebb7fe700 1 -- 172.21.15.122:0/1107494783 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f4ec001a430 con 0x7f4ec4107d10 2024-05-03T17:44:27.065 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.523+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1107494783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 msgr2=0x7f4ec4192e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:27.065 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.523+0000 7f4ecc60b700 1 --2- 172.21.15.122:0/1107494783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4192e80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00117b0 tx=0x7f4ec0011b70 comp rx=0 tx=0).stop 2024-05-03T17:44:27.065 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.523+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1107494783 shutdown_connections 2024-05-03T17:44:27.065 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.523+0000 7f4ecc60b700 1 --2- 172.21.15.122:0/1107494783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ec4107d10 0x7f4ec4192e80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:27.065 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.523+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1107494783 >> 172.21.15.122:0/1107494783 conn(0x7f4ec41032f0 msgr2=0x7f4ec41041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:27.065 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.523+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1107494783 shutdown_connections 2024-05-03T17:44:27.065 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:26.523+0000 7f4ecc60b700 1 -- 172.21.15.122:0/1107494783 wait complete. 2024-05-03T17:44:27.065 INFO:teuthology.orchestra.run.smithi122.stdout:Restarting the monitor... 2024-05-03T17:44:27.334 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:27 smithi122 systemd[1]: Stopping Ceph mon.a for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:44:27.700 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:27 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a[29368]: 2024-05-03T17:44:27.337+0000 7f5828648700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T17:44:27.700 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:27 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a[29368]: 2024-05-03T17:44:27.337+0000 7f5828648700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-03T17:44:27.700 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:27 smithi122 podman[29616]: 2024-05-03 17:44:27.4503436 +0000 UTC m=+0.222430696 container died ddaf75b5e79418168eb8e390167a8a0fea4e520353da94b6741d2fd1657f941f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T17:44:27.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:27 smithi122 podman[29616]: 2024-05-03 17:44:27.820942463 +0000 UTC m=+0.593029555 container cleanup ddaf75b5e79418168eb8e390167a8a0fea4e520353da94b6741d2fd1657f941f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True) 2024-05-03T17:44:27.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:27 smithi122 bash[29616]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a 2024-05-03T17:44:28.404 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 podman[29630]: 2024-05-03 17:44:28.071780734 +0000 UTC m=+0.621293947 container remove ddaf75b5e79418168eb8e390167a8a0fea4e520353da94b6741d2fd1657f941f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-03T17:44:28.404 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 systemd[1]: ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-03T17:44:28.404 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 systemd[1]: Stopped Ceph mon.a for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:44:28.404 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 systemd[1]: Starting Ceph mon.a for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:44:28.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 podman[29725]: 2024-05-03 17:44:28.311790073 +0000 UTC m=+0.017783239 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:44:28.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 podman[29725]: 2024-05-03 17:44:28.434712335 +0000 UTC m=+0.140705470 container create 6c70458e7d5bdd991afecfb7f7c6ee19b0d8e14070af64956377feb47a08e2ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-05-03T17:44:28.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 podman[29725]: 2024-05-03 17:44:28.625180051 +0000 UTC m=+0.331173191 container init 6c70458e7d5bdd991afecfb7f7c6ee19b0d8e14070af64956377feb47a08e2ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux ) 2024-05-03T17:44:28.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 podman[29725]: 2024-05-03 17:44:28.627997014 +0000 UTC m=+0.333990149 container start 6c70458e7d5bdd991afecfb7f7c6ee19b0d8e14070af64956377feb47a08e2ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: set uid:gid to 167:167 (ceph:ceph) 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev), process ceph-mon, pid 2 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: pidfile_write: ignore empty --pid-file 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: load: jerasure load: lrc 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: RocksDB version: 7.9.2 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Git sha 0 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Compile date 2024-05-01 21:12:03 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: DB SUMMARY 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: DB Session ID: DJCKVXOU7T8MKP03KUCE 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: CURRENT file: CURRENT 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: IDENTITY file: IDENTITY 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-03T17:44:28.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104977 ; 2024-05-03T17:44:28.657 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.error_if_exists: 0 2024-05-03T17:44:28.657 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.create_if_missing: 0 2024-05-03T17:44:28.657 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.paranoid_checks: 1 2024-05-03T17:44:28.657 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-03T17:44:28.657 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-03T17:44:28.657 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-03T17:44:28.657 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.env: 0x55e274b82cc0 2024-05-03T17:44:28.657 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.fs: PosixFileSystem 2024-05-03T17:44:28.657 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.info_log: 0x55e2758bbda0 2024-05-03T17:44:28.657 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_file_opening_threads: 16 2024-05-03T17:44:28.657 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.statistics: (nil) 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.use_fsync: 0 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_log_file_size: 0 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.keep_log_file_num: 1000 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.recycle_log_file_num: 0 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.allow_fallocate: 1 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.allow_mmap_reads: 0 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.allow_mmap_writes: 0 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.use_direct_reads: 0 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.create_missing_column_families: 0 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.db_log_dir: 2024-05-03T17:44:28.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.wal_dir: 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.advise_random_on_open: 1 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.db_write_buffer_size: 0 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.write_buffer_manager: 0x55e27588d0e0 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.rate_limiter: (nil) 2024-05-03T17:44:28.659 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-03T17:44:28.660 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.wal_recovery_mode: 2 2024-05-03T17:44:28.660 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.enable_thread_tracking: 0 2024-05-03T17:44:28.660 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.enable_pipelined_write: 0 2024-05-03T17:44:28.660 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.unordered_write: 0 2024-05-03T17:44:28.660 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-03T17:44:28.660 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-03T17:44:28.660 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-03T17:44:28.660 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-03T17:44:28.660 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.row_cache: None 2024-05-03T17:44:28.660 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.wal_filter: None 2024-05-03T17:44:28.660 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.allow_ingest_behind: 0 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.two_write_queues: 0 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.manual_wal_flush: 0 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.wal_compression: 0 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.atomic_flush: 0 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.log_readahead_size: 0 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.best_efforts_recovery: 0 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-03T17:44:28.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.allow_data_in_errors: 0 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.db_host_id: __hostname__ 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_background_jobs: 2 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_background_compactions: -1 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_subcompactions: 1 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_total_wal_size: 0 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-03T17:44:28.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-03T17:44:28.665 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_open_files: -1 2024-05-03T17:44:28.665 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bytes_per_sync: 0 2024-05-03T17:44:28.665 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-03T17:44:28.665 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-03T17:44:28.665 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_readahead_size: 0 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_background_flushes: -1 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Compression algorithms supported: 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: kZSTD supported: 0 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: kXpressCompression supported: 0 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: kLZ4HCCompression supported: 1 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: kZlibCompression supported: 1 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: kSnappyCompression supported: 1 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: kLZ4Compression supported: 1 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: kBZip2Compression supported: 0 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-03T17:44:28.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.merge_operator: 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_filter: None 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_filter_factory: None 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.sst_partitioner_factory: None 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e2758bbf60) 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout: cache_index_and_filter_blocks: 1 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout: pin_top_level_index_and_filter: 1 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_type: 0 2024-05-03T17:44:28.667 INFO:journalctl@ceph.mon.a.smithi122.stdout: data_block_index_type: 0 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_shortening: 1 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: checksum: 4 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: no_block_cache: 0 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache: 0x55e2758bf1f0 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_name: BinnedLRUCache 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_options: 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: capacity : 536870912 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: num_shard_bits : 4 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: strict_capacity_limit : 0 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: high_pri_pool_ratio: 0.000 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_compressed: (nil) 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: persistent_cache: (nil) 2024-05-03T17:44:28.668 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_size: 4096 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_size_deviation: 10 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_restart_interval: 16 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_block_restart_interval: 1 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: metadata_block_size: 4096 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: partition_filters: 0 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: use_delta_encoding: 1 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: filter_policy: bloomfilter 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: whole_key_filtering: 1 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: verify_compression: 0 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: read_amp_bytes_per_bit: 0 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: format_version: 5 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: enable_index_compression: 1 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_align: 0 2024-05-03T17:44:28.669 INFO:journalctl@ceph.mon.a.smithi122.stdout: max_auto_readahead_size: 262144 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout: prepopulate_block_cache: 0 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout: initial_auto_readahead_size: 8192 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout: num_file_reads_for_auto_readahead: 2 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.write_buffer_size: 33554432 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_write_buffer_number: 2 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compression: NoCompression 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bottommost_compression: Disabled 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.prefix_extractor: nullptr 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.num_levels: 7 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-03T17:44:28.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compression_opts.level: 32767 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compression_opts.strategy: 0 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-03T17:44:28.671 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compression_opts.enabled: false 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.target_file_size_base: 67108864 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-03T17:44:28.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.arena_block_size: 1048576 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.disable_auto_compactions: 0 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-03T17:44:28.673 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.inplace_update_support: 0 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.bloom_locality: 0 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.max_successive_merges: 0 2024-05-03T17:44:28.674 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.paranoid_file_checks: 0 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.force_consistency_checks: 1 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.report_bg_io_stats: 0 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.ttl: 2592000 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.enable_blob_files: false 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.min_blob_size: 0 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.blob_file_size: 268435456 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-03T17:44:28.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.blob_file_starting_level: 0 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9574b022-57aa-4a14-b38c-68d9ec15d3f9 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714758268656725, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714758268657744, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100029, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 97915, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14088, "raw_average_key_size": 49, "raw_value_size": 90240, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714758268, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9574b022-57aa-4a14-b38c-68d9ec15d3f9", "db_session_id": "DJCKVXOU7T8MKP03KUCE", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714758268657802, "job": 1, "event": "recovery_finished"} 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e2759dc000 2024-05-03T17:44:28.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: DB pointer 0x55e2759c8000 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** DB Stats ** 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-03T17:44:28.677 INFO:journalctl@ceph.mon.a.smithi122.stdout: L0 2/0 99.54 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: Sum 2/0 99.54 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 149.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-03T17:44:28.678 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative compaction: 0.00 GB write, 32.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval compaction: 0.00 GB write, 32.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: Block cache BinnedLRUCache@0x55e2758bf1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: starting mon.a rank 0 at public addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] at bind addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: mon.a@-1(???) e1 preinit fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: mon.a@-1(???).mds e1 new map 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: mon.a@-1(???).mds e1 print_map 2024-05-03T17:44:28.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: e1 2024-05-03T17:44:28.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T17:44:28.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T17:44:28.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: legacy client fscid: -1 2024-05-03T17:44:28.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-05-03T17:44:28.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: No filesystems configured 2024-05-03T17:44:28.680 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T17:44:28.680 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T17:44:28.680 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T17:44:28.680 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T17:44:28.680 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-03T17:44:28.726 INFO:teuthology.orchestra.run.smithi122.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-03T17:44:28.947 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T17:44:28.947 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: monmap epoch 1 2024-05-03T17:44:28.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:28.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: last_changed 2024-05-03T17:44:10.382958+0000 2024-05-03T17:44:28.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: created 2024-05-03T17:44:10.382958+0000 2024-05-03T17:44:28.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: min_mon_release 19 (squid) 2024-05-03T17:44:28.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: election_strategy: 1 2024-05-03T17:44:28.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-05-03T17:44:28.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: fsmap 2024-05-03T17:44:28.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: osdmap e1: 0 total, 0 up, 0 in 2024-05-03T17:44:28.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 ceph-mon[29744]: mgrmap e1: no daemons active 2024-05-03T17:44:28.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 bash[29725]: 6c70458e7d5bdd991afecfb7f7c6ee19b0d8e14070af64956377feb47a08e2ef 2024-05-03T17:44:28.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:28 smithi122 systemd[1]: Started Ceph mon.a for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:44:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:30 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/68971532' entity='client.admin' 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.162+0000 7f162f063700 1 Processor -- start 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.162+0000 7f162f063700 1 -- start start 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.162+0000 7f162f063700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162810bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.163+0000 7f162f063700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f162807d730 con 0x7f16281098a0 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.163+0000 7f162cdff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162810bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.163+0000 7f162cdff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162810bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36888/0 (socket says 172.21.15.122:36888) 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.163+0000 7f162cdff700 1 -- 172.21.15.122:0/582536203 learned_addr learned my addr 172.21.15.122:0/582536203 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.163+0000 7f162cdff700 1 -- 172.21.15.122:0/582536203 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162810c1d0 con 0x7f16281098a0 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.163+0000 7f162cdff700 1 --2- 172.21.15.122:0/582536203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162810bc90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f161c0098a0 tx=0x7f161c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9569ccc6549ca82 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.163+0000 7f16277fe700 1 -- 172.21.15.122:0/582536203 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c004160 con 0x7f16281098a0 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.163+0000 7f16277fe700 1 -- 172.21.15.122:0/582536203 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f161c004300 con 0x7f16281098a0 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.163+0000 7f16277fe700 1 -- 172.21.15.122:0/582536203 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c011af0 con 0x7f16281098a0 2024-05-03T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.164+0000 7f162f063700 1 -- 172.21.15.122:0/582536203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 msgr2=0x7f162810bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.164+0000 7f162f063700 1 --2- 172.21.15.122:0/582536203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162810bc90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f161c0098a0 tx=0x7f161c009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.164+0000 7f162f063700 1 -- 172.21.15.122:0/582536203 shutdown_connections 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.164+0000 7f162f063700 1 --2- 172.21.15.122:0/582536203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162810bc90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.164+0000 7f162f063700 1 -- 172.21.15.122:0/582536203 >> 172.21.15.122:0/582536203 conn(0x7f16281032f0 msgr2=0x7f1628105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.164+0000 7f162f063700 1 -- 172.21.15.122:0/582536203 shutdown_connections 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.164+0000 7f162f063700 1 -- 172.21.15.122:0/582536203 wait complete. 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.165+0000 7f162f063700 1 Processor -- start 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.165+0000 7f162f063700 1 -- start start 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.165+0000 7f162f063700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.165+0000 7f162f063700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f162810d120 con 0x7f16281098a0 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.165+0000 7f162cdff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.165+0000 7f162cdff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36900/0 (socket says 172.21.15.122:36900) 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.165+0000 7f162cdff700 1 -- 172.21.15.122:0/68971532 learned_addr learned my addr 172.21.15.122:0/68971532 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.165+0000 7f162cdff700 1 -- 172.21.15.122:0/68971532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162819d220 con 0x7f16281098a0 2024-05-03T17:44:31.085 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.165+0000 7f162cdff700 1 --2- 172.21.15.122:0/68971532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162819cce0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f161c009fb0 tx=0x7f161c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.166+0000 7f1625ffb700 1 -- 172.21.15.122:0/68971532 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c0036a0 con 0x7f16281098a0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.166+0000 7f1625ffb700 1 -- 172.21.15.122:0/68971532 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f161c01a430 con 0x7f16281098a0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.166+0000 7f1625ffb700 1 -- 172.21.15.122:0/68971532 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161c019550 con 0x7f16281098a0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.166+0000 7f162f063700 1 -- 172.21.15.122:0/68971532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162819d510 con 0x7f16281098a0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.166+0000 7f162f063700 1 -- 172.21.15.122:0/68971532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16281a0190 con 0x7f16281098a0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.166+0000 7f1625ffb700 1 -- 172.21.15.122:0/68971532 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f161c019b90 con 0x7f16281098a0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.166+0000 7f1625ffb700 1 -- 172.21.15.122:0/68971532 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f161c024730 con 0x7f16281098a0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.167+0000 7f162f063700 1 -- 172.21.15.122:0/68971532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162806a5c0 con 0x7f16281098a0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.169+0000 7f1625ffb700 1 -- 172.21.15.122:0/68971532 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f161c01a5e0 con 0x7f16281098a0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.236+0000 7f162f063700 1 -- 172.21.15.122:0/68971532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f16281a0360 con 0x7f16281098a0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.238+0000 7f1625ffb700 1 -- 172.21.15.122:0/68971532 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f161c02dcb0 con 0x7f16281098a0 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.239+0000 7f162f063700 1 -- 172.21.15.122:0/68971532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 msgr2=0x7f162819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:31.086 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.239+0000 7f162f063700 1 --2- 172.21.15.122:0/68971532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162819cce0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f161c009fb0 tx=0x7f161c011940 comp rx=0 tx=0).stop 2024-05-03T17:44:31.087 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.239+0000 7f162f063700 1 -- 172.21.15.122:0/68971532 shutdown_connections 2024-05-03T17:44:31.087 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.239+0000 7f162f063700 1 --2- 172.21.15.122:0/68971532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16281098a0 0x7f162819cce0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:31.087 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.239+0000 7f162f063700 1 -- 172.21.15.122:0/68971532 >> 172.21.15.122:0/68971532 conn(0x7f16281032f0 msgr2=0x7f1628103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:31.087 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.239+0000 7f162f063700 1 -- 172.21.15.122:0/68971532 shutdown_connections 2024-05-03T17:44:31.087 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:29.239+0000 7f162f063700 1 -- 172.21.15.122:0/68971532 wait complete. 2024-05-03T17:44:31.087 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-03T17:44:31.117 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-03T17:44:31.118 INFO:teuthology.orchestra.run.smithi122.stdout:Creating mgr... 2024-05-03T17:44:31.118 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-03T17:44:31.121 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-03T17:44:31.367 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.x 2024-05-03T17:44:31.367 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-03T17:44:31.521 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a46fc846-0974-11ef-bc97-c7b262605968.target.wants/ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-a46fc846-0974-11ef-bc97-c7b262605968@.service. 2024-05-03T17:44:31.710 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:31 smithi122 systemd[1]: Starting Ceph mgr.x for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:44:31.998 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:31 smithi122 podman[29958]: 2024-05-03 17:44:31.741778885 +0000 UTC m=+0.098115939 container create fec6baba9e62d64a109cf367889b544c69d9c180975c19b11b3c41f72305da43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T17:44:31.998 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:31 smithi122 podman[29958]: 2024-05-03 17:44:31.662199124 +0000 UTC m=+0.018536196 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:44:31.998 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:31 smithi122 podman[29958]: 2024-05-03 17:44:31.952403847 +0000 UTC m=+0.308740902 container init fec6baba9e62d64a109cf367889b544c69d9c180975c19b11b3c41f72305da43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-03T17:44:31.998 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:31 smithi122 podman[29958]: 2024-05-03 17:44:31.955086337 +0000 UTC m=+0.311423392 container start fec6baba9e62d64a109cf367889b544c69d9c180975c19b11b3c41f72305da43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-05-03T17:44:31.998 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:32 smithi122 bash[29958]: fec6baba9e62d64a109cf367889b544c69d9c180975c19b11b3c41f72305da43 2024-05-03T17:44:32.010 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T17:44:32.011 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-05-03T17:44:32.018 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T17:44:32.018 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-05-03T17:44:32.018 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-05-03T17:44:32.018 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to enable service . firewalld.service is not available 2024-05-03T17:44:32.026 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T17:44:32.026 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-05-03T17:44:32.034 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T17:44:32.034 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-05-03T17:44:32.034 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-05-03T17:44:32.034 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-03T17:44:32.034 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr to start... 2024-05-03T17:44:32.034 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr... 2024-05-03T17:44:32.294 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:32 smithi122 systemd[1]: Started Ceph mgr.x for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:44:32.295 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:32 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:32.109+0000 7fa6095a9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T17:44:32.295 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:32 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:32.298+0000 7fa6095a9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T17:44:32.553 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:32 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:32.403+0000 7fa6095a9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T17:44:32.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:32 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1215806746' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T17:44:32.831 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:32 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:32.747+0000 7fa6095a9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T17:44:32.831 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:32 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:32.835+0000 7fa6095a9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T17:44:33.185 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:33.185 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-05-03T17:44:33.185 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "a46fc846-0974-11ef-bc97-c7b262605968", 2024-05-03T17:44:33.185 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T17:44:33.185 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T17:44:33.185 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T17:44:33.185 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:33.187 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T17:44:33.187 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:33.187 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T17:44:33.187 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T17:44:33.187 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T17:44:33.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T17:44:33.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T17:44:17.416098+0000", 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.439+0000 7f9a30eb7700 1 Processor -- start 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.439+0000 7f9a30eb7700 1 -- start start 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.440+0000 7f9a30eb7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c10bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.440+0000 7f9a30eb7700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c07d670 con 0x7f9a2c109950 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.440+0000 7f9a2a59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c10bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.440+0000 7f9a2a59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c10bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36912/0 (socket says 172.21.15.122:36912) 2024-05-03T17:44:33.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.440+0000 7f9a2a59c700 1 -- 172.21.15.122:0/4016213626 learned_addr learned my addr 172.21.15.122:0/4016213626 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.440+0000 7f9a2a59c700 1 -- 172.21.15.122:0/4016213626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c10c280 con 0x7f9a2c109950 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.441+0000 7f9a2a59c700 1 --2- 172.21.15.122:0/4016213626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c10bd40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0098a0 tx=0x7f9a1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b71054ec4645a3b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.441+0000 7f9a2959a700 1 -- 172.21.15.122:0/4016213626 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c004160 con 0x7f9a2c109950 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.441+0000 7f9a2959a700 1 -- 172.21.15.122:0/4016213626 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9a1c004300 con 0x7f9a2c109950 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.441+0000 7f9a2959a700 1 -- 172.21.15.122:0/4016213626 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c011af0 con 0x7f9a2c109950 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.441+0000 7f9a30eb7700 1 -- 172.21.15.122:0/4016213626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 msgr2=0x7f9a2c10bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.441+0000 7f9a30eb7700 1 --2- 172.21.15.122:0/4016213626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c10bd40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0098a0 tx=0x7f9a1c009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.442+0000 7f9a30eb7700 1 -- 172.21.15.122:0/4016213626 shutdown_connections 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.442+0000 7f9a30eb7700 1 --2- 172.21.15.122:0/4016213626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c10bd40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.442+0000 7f9a30eb7700 1 -- 172.21.15.122:0/4016213626 >> 172.21.15.122:0/4016213626 conn(0x7f9a2c1032c0 msgr2=0x7f9a2c1056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.442+0000 7f9a30eb7700 1 -- 172.21.15.122:0/4016213626 shutdown_connections 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.442+0000 7f9a30eb7700 1 -- 172.21.15.122:0/4016213626 wait complete. 2024-05-03T17:44:33.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.442+0000 7f9a30eb7700 1 Processor -- start 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.442+0000 7f9a30eb7700 1 -- start start 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a30eb7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a30eb7700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c10d180 con 0x7f9a2c109950 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a2a59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a2a59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36914/0 (socket says 172.21.15.122:36914) 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a2a59c700 1 -- 172.21.15.122:0/1215806746 learned_addr learned my addr 172.21.15.122:0/1215806746 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a2a59c700 1 -- 172.21.15.122:0/1215806746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c19d2e0 con 0x7f9a2c109950 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a2a59c700 1 --2- 172.21.15.122:0/1215806746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c19cda0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c011ac0 tx=0x7f9a1c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a1b7fe700 1 -- 172.21.15.122:0/1215806746 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c003970 con 0x7f9a2c109950 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a1b7fe700 1 -- 172.21.15.122:0/1215806746 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9a1c003b10 con 0x7f9a2c109950 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a30eb7700 1 -- 172.21.15.122:0/1215806746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c19d570 con 0x7f9a2c109950 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a1b7fe700 1 -- 172.21.15.122:0/1215806746 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c024610 con 0x7f9a2c109950 2024-05-03T17:44:33.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.443+0000 7f9a30eb7700 1 -- 172.21.15.122:0/1215806746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c1a0220 con 0x7f9a2c109950 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.444+0000 7f9a1b7fe700 1 -- 172.21.15.122:0/1215806746 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9a1c024c20 con 0x7f9a2c109950 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.444+0000 7f9a1b7fe700 1 -- 172.21.15.122:0/1215806746 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9a1c019700 con 0x7f9a2c109950 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.445+0000 7f9a30eb7700 1 -- 172.21.15.122:0/1215806746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a2c06a5c0 con 0x7f9a2c109950 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.447+0000 7f9a1b7fe700 1 -- 172.21.15.122:0/1215806746 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9a1c043e50 con 0x7f9a2c109950 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.541+0000 7f9a30eb7700 1 -- 172.21.15.122:0/1215806746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9a2c1a03f0 con 0x7f9a2c109950 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.542+0000 7f9a1b7fe700 1 -- 172.21.15.122:0/1215806746 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9a2c1a03f0 con 0x7f9a2c109950 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.543+0000 7f9a30eb7700 1 -- 172.21.15.122:0/1215806746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 msgr2=0x7f9a2c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.543+0000 7f9a30eb7700 1 --2- 172.21.15.122:0/1215806746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c19cda0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c011ac0 tx=0x7f9a1c011ba0 comp rx=0 tx=0).stop 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.543+0000 7f9a30eb7700 1 -- 172.21.15.122:0/1215806746 shutdown_connections 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.543+0000 7f9a30eb7700 1 --2- 172.21.15.122:0/1215806746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a2c109950 0x7f9a2c19cda0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.543+0000 7f9a30eb7700 1 -- 172.21.15.122:0/1215806746 >> 172.21.15.122:0/1215806746 conn(0x7f9a2c1032c0 msgr2=0x7f9a2c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.543+0000 7f9a30eb7700 1 -- 172.21.15.122:0/1215806746 shutdown_connections 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:32.543+0000 7f9a30eb7700 1 -- 172.21.15.122:0/1215806746 wait complete. 2024-05-03T17:44:33.200 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (1/15)... 2024-05-03T17:44:34.351 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:34.017+0000 7fa6095a9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T17:44:34.351 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:34.183+0000 7fa6095a9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T17:44:34.351 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:34.354+0000 7fa6095a9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T17:44:34.949 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:34.655+0000 7fa6095a9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T17:44:34.949 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:34.749+0000 7fa6095a9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T17:44:34.949 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:34.953+0000 7fa6095a9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T17:44:35.210 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:35 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:35.146+0000 7fa6095a9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T17:44:35.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:35 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2255280446' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T17:44:35.961 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:35 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:35.771+0000 7fa6095a9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T17:44:36.350 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:36 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:36.000+0000 7fa6095a9200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T17:44:36.710 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:36 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:36.421+0000 7fa6095a9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T17:44:36.711 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:36 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:36.517+0000 7fa6095a9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T17:44:36.819 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:36.819 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-05-03T17:44:36.819 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "a46fc846-0974-11ef-bc97-c7b262605968", 2024-05-03T17:44:36.819 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T17:44:36.819 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T17:44:17.416098+0000", 2024-05-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-05-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.585+0000 7f65eda94700 1 Processor -- start 2024-05-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.585+0000 7f65eda94700 1 -- start start 2024-05-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.585+0000 7f65eda94700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e807dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.585+0000 7f65eda94700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f65e807e190 con 0x7f65e807f760 2024-05-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.586+0000 7f65e77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e807dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.586+0000 7f65e77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e807dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36916/0 (socket says 172.21.15.122:36916) 2024-05-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.586+0000 7f65e77fe700 1 -- 172.21.15.122:0/2882877783 learned_addr learned my addr 172.21.15.122:0/2882877783 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.586+0000 7f65e77fe700 1 -- 172.21.15.122:0/2882877783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e807e310 con 0x7f65e807f760 2024-05-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.586+0000 7f65e77fe700 1 --2- 172.21.15.122:0/2882877783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e807dbc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0098a0 tx=0x7f65dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3390ee8291f2716a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.586+0000 7f65e67fc700 1 -- 172.21.15.122:0/2882877783 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65dc004160 con 0x7f65e807f760 2024-05-03T17:44:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.586+0000 7f65e67fc700 1 -- 172.21.15.122:0/2882877783 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f65dc004300 con 0x7f65e807f760 2024-05-03T17:44:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.586+0000 7f65e67fc700 1 -- 172.21.15.122:0/2882877783 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65dc011af0 con 0x7f65e807f760 2024-05-03T17:44:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.587+0000 7f65eda94700 1 -- 172.21.15.122:0/2882877783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 msgr2=0x7f65e807dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.587+0000 7f65eda94700 1 --2- 172.21.15.122:0/2882877783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e807dbc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0098a0 tx=0x7f65dc009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.587+0000 7f65eda94700 1 -- 172.21.15.122:0/2882877783 shutdown_connections 2024-05-03T17:44:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.587+0000 7f65eda94700 1 --2- 172.21.15.122:0/2882877783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e807dbc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.587+0000 7f65eda94700 1 -- 172.21.15.122:0/2882877783 >> 172.21.15.122:0/2882877783 conn(0x7f65e81043d0 msgr2=0x7f65e81067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.587+0000 7f65eda94700 1 -- 172.21.15.122:0/2882877783 shutdown_connections 2024-05-03T17:44:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.587+0000 7f65eda94700 1 -- 172.21.15.122:0/2882877783 wait complete. 2024-05-03T17:44:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.588+0000 7f65eda94700 1 Processor -- start 2024-05-03T17:44:36.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.588+0000 7f65eda94700 1 -- start start 2024-05-03T17:44:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.588+0000 7f65eda94700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.588+0000 7f65eda94700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f65e810f3a0 con 0x7f65e807f760 2024-05-03T17:44:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.588+0000 7f65e77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.588+0000 7f65e77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36920/0 (socket says 172.21.15.122:36920) 2024-05-03T17:44:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.588+0000 7f65e77fe700 1 -- 172.21.15.122:0/2255280446 learned_addr learned my addr 172.21.15.122:0/2255280446 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.588+0000 7f65e77fe700 1 -- 172.21.15.122:0/2255280446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e819f4c0 con 0x7f65e807f760 2024-05-03T17:44:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.588+0000 7f65e77fe700 1 --2- 172.21.15.122:0/2255280446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e819ef80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009fb0 tx=0x7f65dc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.589+0000 7f65e4ff9700 1 -- 172.21.15.122:0/2255280446 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65dc0036a0 con 0x7f65e807f760 2024-05-03T17:44:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.589+0000 7f65e4ff9700 1 -- 172.21.15.122:0/2255280446 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f65dc01a430 con 0x7f65e807f760 2024-05-03T17:44:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.589+0000 7f65eda94700 1 -- 172.21.15.122:0/2255280446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e819f750 con 0x7f65e807f760 2024-05-03T17:44:36.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.589+0000 7f65e4ff9700 1 -- 172.21.15.122:0/2255280446 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65dc019550 con 0x7f65e807f760 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.589+0000 7f65eda94700 1 -- 172.21.15.122:0/2255280446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e81a2440 con 0x7f65e807f760 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.589+0000 7f65e4ff9700 1 -- 172.21.15.122:0/2255280446 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f65dc019be0 con 0x7f65e807f760 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.589+0000 7f65e4ff9700 1 -- 172.21.15.122:0/2255280446 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f65dc024730 con 0x7f65e807f760 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.590+0000 7f65eda94700 1 -- 172.21.15.122:0/2255280446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e806a5c0 con 0x7f65e807f760 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.592+0000 7f65e4ff9700 1 -- 172.21.15.122:0/2255280446 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f65dc043e50 con 0x7f65e807f760 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.664+0000 7f65eda94700 1 -- 172.21.15.122:0/2255280446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f65e81a2980 con 0x7f65e807f760 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.664+0000 7f65e4ff9700 1 -- 172.21.15.122:0/2255280446 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f65dc032030 con 0x7f65e807f760 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.665+0000 7f65eda94700 1 -- 172.21.15.122:0/2255280446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 msgr2=0x7f65e819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.665+0000 7f65eda94700 1 --2- 172.21.15.122:0/2255280446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e819ef80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009fb0 tx=0x7f65dc011940 comp rx=0 tx=0).stop 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.665+0000 7f65eda94700 1 -- 172.21.15.122:0/2255280446 shutdown_connections 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.665+0000 7f65eda94700 1 --2- 172.21.15.122:0/2255280446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65e807f760 0x7f65e819ef80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:36.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.665+0000 7f65eda94700 1 -- 172.21.15.122:0/2255280446 >> 172.21.15.122:0/2255280446 conn(0x7f65e81043d0 msgr2=0x7f65e8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:36.829 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.665+0000 7f65eda94700 1 -- 172.21.15.122:0/2255280446 shutdown_connections 2024-05-03T17:44:36.829 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:35.665+0000 7f65eda94700 1 -- 172.21.15.122:0/2255280446 wait complete. 2024-05-03T17:44:36.829 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (2/15)... 2024-05-03T17:44:37.210 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:36 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:36.886+0000 7fa6095a9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T17:44:37.559 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:37 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:37.236+0000 7fa6095a9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T17:44:37.559 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:37 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:37.326+0000 7fa6095a9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T17:44:37.960 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:37 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:37.562+0000 7fa6095a9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T17:44:37.961 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:37 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:37.692+0000 7fa6095a9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T17:44:38.460 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:37 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:37.990+0000 7fa6095a9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T17:44:38.461 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:38 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:38.103+0000 7fa6095a9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T17:44:38.461 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:38 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:38.192+0000 7fa6095a9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T17:44:39.030 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:38 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:38.781+0000 7fa6095a9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T17:44:39.030 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:39 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:38.999+0000 7fa6095a9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T17:44:39.301 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: Activating manager daemon x 2024-05-03T17:44:39.301 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: mgrmap e2: x(active, starting, since 0.00313815s) 2024-05-03T17:44:39.301 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: from='mgr.14100 172.21.15.122:0/3230254611' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T17:44:39.301 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: from='mgr.14100 172.21.15.122:0/3230254611' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T17:44:39.302 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: from='mgr.14100 172.21.15.122:0/3230254611' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T17:44:39.302 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: from='mgr.14100 172.21.15.122:0/3230254611' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:44:39.302 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: from='mgr.14100 172.21.15.122:0/3230254611' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T17:44:39.302 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: Manager daemon x is now available 2024-05-03T17:44:39.302 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: from='mgr.14100 172.21.15.122:0/3230254611' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T17:44:39.302 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: from='mgr.14100 172.21.15.122:0/3230254611' entity='mgr.x' 2024-05-03T17:44:39.302 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: from='mgr.14100 172.21.15.122:0/3230254611' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T17:44:39.302 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: from='mgr.14100 172.21.15.122:0/3230254611' entity='mgr.x' 2024-05-03T17:44:39.302 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:39 smithi122 ceph-mon[29744]: from='mgr.14100 172.21.15.122:0/3230254611' entity='mgr.x' 2024-05-03T17:44:40.023 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:40.023 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-05-03T17:44:40.023 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "a46fc846-0974-11ef-bc97-c7b262605968", 2024-05-03T17:44:40.023 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T17:44:40.023 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T17:44:40.023 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T17:44:40.024 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T17:44:40.025 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T17:44:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T17:44:17.416098+0000", 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.216+0000 7f7599bf0700 1 Processor -- start 2024-05-03T17:44:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.216+0000 7f7599bf0700 1 -- start start 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.216+0000 7f7599bf0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f7594109230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.216+0000 7f7599bf0700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7594109770 con 0x7f7594108e10 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.217+0000 7f75937fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f7594109230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.217+0000 7f75937fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f7594109230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57064/0 (socket says 172.21.15.122:57064) 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.217+0000 7f75937fe700 1 -- 172.21.15.122:0/3797224524 learned_addr learned my addr 172.21.15.122:0/3797224524 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.217+0000 7f75937fe700 1 -- 172.21.15.122:0/3797224524 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7594109f70 con 0x7f7594108e10 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.217+0000 7f75937fe700 1 --2- 172.21.15.122:0/3797224524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f7594109230 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f75840098a0 tx=0x7f7584009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349f3ec433276b94 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.217+0000 7f7592ffd700 1 -- 172.21.15.122:0/3797224524 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7584004160 con 0x7f7594108e10 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.217+0000 7f7592ffd700 1 -- 172.21.15.122:0/3797224524 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7584004300 con 0x7f7594108e10 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.217+0000 7f7592ffd700 1 -- 172.21.15.122:0/3797224524 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7584011af0 con 0x7f7594108e10 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.218+0000 7f7599bf0700 1 -- 172.21.15.122:0/3797224524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 msgr2=0x7f7594109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.218+0000 7f7599bf0700 1 --2- 172.21.15.122:0/3797224524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f7594109230 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f75840098a0 tx=0x7f7584009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.218+0000 7f7599bf0700 1 -- 172.21.15.122:0/3797224524 shutdown_connections 2024-05-03T17:44:40.028 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.218+0000 7f7599bf0700 1 --2- 172.21.15.122:0/3797224524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f7594109230 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.218+0000 7f7599bf0700 1 -- 172.21.15.122:0/3797224524 >> 172.21.15.122:0/3797224524 conn(0x7f75941043d0 msgr2=0x7f75941067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.218+0000 7f7599bf0700 1 -- 172.21.15.122:0/3797224524 shutdown_connections 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.218+0000 7f7599bf0700 1 -- 172.21.15.122:0/3797224524 wait complete. 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.218+0000 7f7599bf0700 1 Processor -- start 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.218+0000 7f7599bf0700 1 -- start start 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f7599bf0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f759419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f7599bf0700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f759410d3b0 con 0x7f7594108e10 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f75937fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f759419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f75937fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f759419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57076/0 (socket says 172.21.15.122:57076) 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f75937fe700 1 -- 172.21.15.122:0/1796666036 learned_addr learned my addr 172.21.15.122:0/1796666036 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f75937fe700 1 -- 172.21.15.122:0/1796666036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759419d260 con 0x7f7594108e10 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f75937fe700 1 --2- 172.21.15.122:0/1796666036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f759419cd20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7584011ac0 tx=0x7f7584011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f75917fa700 1 -- 172.21.15.122:0/1796666036 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7584003970 con 0x7f7594108e10 2024-05-03T17:44:40.029 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f75917fa700 1 -- 172.21.15.122:0/1796666036 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7584003b10 con 0x7f7594108e10 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f75917fa700 1 -- 172.21.15.122:0/1796666036 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7584024610 con 0x7f7594108e10 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f7599bf0700 1 -- 172.21.15.122:0/1796666036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759419d4f0 con 0x7f7594108e10 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.219+0000 7f7599bf0700 1 -- 172.21.15.122:0/1796666036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75941a01d0 con 0x7f7594108e10 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.220+0000 7f75917fa700 1 -- 172.21.15.122:0/1796666036 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 2) ==== 49978+0+0 (secure 0 0 0) 0x7f7584024840 con 0x7f7594108e10 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.220+0000 7f7599bf0700 1 -- 172.21.15.122:0/1796666036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759406a5c0 con 0x7f7594108e10 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.221+0000 7f75917fa700 1 -- 172.21.15.122:0/1796666036 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f75840535c0 con 0x7f7594108e10 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.223+0000 7f75917fa700 1 -- 172.21.15.122:0/1796666036 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f758402c350 con 0x7f7594108e10 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.293+0000 7f7599bf0700 1 -- 172.21.15.122:0/1796666036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f75941a03a0 con 0x7f7594108e10 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.294+0000 7f75917fa700 1 -- 172.21.15.122:0/1796666036 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f75941a03a0 con 0x7f7594108e10 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.294+0000 7f7599bf0700 1 -- 172.21.15.122:0/1796666036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 msgr2=0x7f759419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.294+0000 7f7599bf0700 1 --2- 172.21.15.122:0/1796666036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f759419cd20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7584011ac0 tx=0x7f7584011ba0 comp rx=0 tx=0).stop 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.295+0000 7f7599bf0700 1 -- 172.21.15.122:0/1796666036 shutdown_connections 2024-05-03T17:44:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.295+0000 7f7599bf0700 1 --2- 172.21.15.122:0/1796666036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7594108e10 0x7f759419cd20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:40.031 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.295+0000 7f7599bf0700 1 -- 172.21.15.122:0/1796666036 >> 172.21.15.122:0/1796666036 conn(0x7f75941043d0 msgr2=0x7f7594104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:40.031 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.295+0000 7f7599bf0700 1 -- 172.21.15.122:0/1796666036 shutdown_connections 2024-05-03T17:44:40.031 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:39.295+0000 7f7599bf0700 1 -- 172.21.15.122:0/1796666036 wait complete. 2024-05-03T17:44:40.031 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (3/15)... 2024-05-03T17:44:40.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:40 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1796666036' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T17:44:40.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:40 smithi122 ceph-mon[29744]: mgrmap e3: x(active, since 1.00769s) 2024-05-03T17:44:42.347 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:42 smithi122 ceph-mon[29744]: mgrmap e4: x(active, since 2s) 2024-05-03T17:44:43.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "a46fc846-0974-11ef-bc97-c7b262605968", 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:43.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T17:44:43.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T17:44:43.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T17:44:17.416098+0000", 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.451+0000 7f52a8867700 1 Processor -- start 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.451+0000 7f52a8867700 1 -- start start 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.451+0000 7f52a8867700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a4107b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.452+0000 7f52a8867700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52a407ce80 con 0x7f52a41076f0 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.452+0000 7f52a259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a4107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.452+0000 7f52a259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a4107b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57078/0 (socket says 172.21.15.122:57078) 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.452+0000 7f52a259c700 1 -- 172.21.15.122:0/4178523738 learned_addr learned my addr 172.21.15.122:0/4178523738 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.452+0000 7f52a259c700 1 -- 172.21.15.122:0/4178523738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a4108050 con 0x7f52a41076f0 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.452+0000 7f52a259c700 1 --2- 172.21.15.122:0/4178523738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a4107b10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5294009e80 tx=0x7f529400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce72dfe875ee324 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:43.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.452+0000 7f52a159a700 1 -- 172.21.15.122:0/4178523738 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5294017070 con 0x7f52a41076f0 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.452+0000 7f52a159a700 1 -- 172.21.15.122:0/4178523738 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f529400b8c0 con 0x7f52a41076f0 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.452+0000 7f52a159a700 1 -- 172.21.15.122:0/4178523738 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5294004490 con 0x7f52a41076f0 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.453+0000 7f52a8867700 1 -- 172.21.15.122:0/4178523738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 msgr2=0x7f52a4107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.453+0000 7f52a8867700 1 --2- 172.21.15.122:0/4178523738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a4107b10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5294009e80 tx=0x7f529400b330 comp rx=0 tx=0).stop 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.453+0000 7f52a8867700 1 -- 172.21.15.122:0/4178523738 shutdown_connections 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.453+0000 7f52a8867700 1 --2- 172.21.15.122:0/4178523738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a4107b10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.453+0000 7f52a8867700 1 -- 172.21.15.122:0/4178523738 >> 172.21.15.122:0/4178523738 conn(0x7f52a41032f0 msgr2=0x7f52a4105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.453+0000 7f52a8867700 1 -- 172.21.15.122:0/4178523738 shutdown_connections 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.453+0000 7f52a8867700 1 -- 172.21.15.122:0/4178523738 wait complete. 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.454+0000 7f52a8867700 1 Processor -- start 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.454+0000 7f52a8867700 1 -- start start 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.454+0000 7f52a8867700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.454+0000 7f52a8867700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52a4109080 con 0x7f52a41076f0 2024-05-03T17:44:43.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.454+0000 7f52a259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.454+0000 7f52a259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57086/0 (socket says 172.21.15.122:57086) 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.454+0000 7f52a259c700 1 -- 172.21.15.122:0/705799104 learned_addr learned my addr 172.21.15.122:0/705799104 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.454+0000 7f52a259c700 1 -- 172.21.15.122:0/705799104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a419f4b0 con 0x7f52a41076f0 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.455+0000 7f52a259c700 1 --2- 172.21.15.122:0/705799104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a419ef70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f529400b7b0 tx=0x7f529400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.455+0000 7f529b7fe700 1 -- 172.21.15.122:0/705799104 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5294020020 con 0x7f52a41076f0 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.455+0000 7f529b7fe700 1 -- 172.21.15.122:0/705799104 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5294012430 con 0x7f52a41076f0 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.455+0000 7f529b7fe700 1 -- 172.21.15.122:0/705799104 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5294017040 con 0x7f52a41076f0 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.455+0000 7f52a8867700 1 -- 172.21.15.122:0/705799104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a419f740 con 0x7f52a41076f0 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.455+0000 7f52a8867700 1 -- 172.21.15.122:0/705799104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a41a2430 con 0x7f52a41076f0 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.456+0000 7f529b7fe700 1 -- 172.21.15.122:0/705799104 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f5294023020 con 0x7f52a41076f0 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.456+0000 7f52a8867700 1 -- 172.21.15.122:0/705799104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a406a5c0 con 0x7f52a41076f0 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.456+0000 7f529b7fe700 1 --2- 172.21.15.122:0/705799104 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f529003e420 0x7f52900408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.456+0000 7f529b7fe700 1 -- 172.21.15.122:0/705799104 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f529400f050 con 0x7f52a41076f0 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.456+0000 7f52a1d9b700 1 --2- 172.21.15.122:0/705799104 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f529003e420 0x7f52900408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.457+0000 7f52a1d9b700 1 --2- 172.21.15.122:0/705799104 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f529003e420 0x7f52900408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f528c006fd0 tx=0x7f528c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.461+0000 7f529b7fe700 1 -- 172.21.15.122:0/705799104 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f529402a020 con 0x7f52a41076f0 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.758+0000 7f52a8867700 1 -- 172.21.15.122:0/705799104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f52a402e880 con 0x7f52a41076f0 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.758+0000 7f529b7fe700 1 -- 172.21.15.122:0/705799104 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5294053080 con 0x7f52a41076f0 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.760+0000 7f52a8867700 1 -- 172.21.15.122:0/705799104 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f529003e420 msgr2=0x7f52900408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.760+0000 7f52a8867700 1 --2- 172.21.15.122:0/705799104 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f529003e420 0x7f52900408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f528c006fd0 tx=0x7f528c006e40 comp rx=0 tx=0).stop 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.760+0000 7f52a8867700 1 -- 172.21.15.122:0/705799104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 msgr2=0x7f52a419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.760+0000 7f52a8867700 1 --2- 172.21.15.122:0/705799104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a419ef70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f529400b7b0 tx=0x7f529400be20 comp rx=0 tx=0).stop 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.760+0000 7f52a8867700 1 -- 172.21.15.122:0/705799104 shutdown_connections 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.760+0000 7f52a8867700 1 --2- 172.21.15.122:0/705799104 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f529003e420 0x7f52900408e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.760+0000 7f52a8867700 1 --2- 172.21.15.122:0/705799104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a41076f0 0x7f52a419ef70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.760+0000 7f52a8867700 1 -- 172.21.15.122:0/705799104 >> 172.21.15.122:0/705799104 conn(0x7f52a41032f0 msgr2=0x7f52a4195c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:43.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.761+0000 7f52a8867700 1 -- 172.21.15.122:0/705799104 shutdown_connections 2024-05-03T17:44:43.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:42.761+0000 7f52a8867700 1 -- 172.21.15.122:0/705799104 wait complete. 2024-05-03T17:44:43.297 INFO:teuthology.orchestra.run.smithi122.stdout:mgr is available 2024-05-03T17:44:43.404 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:43 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/705799104' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T17:44:44.448 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:44 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2714705037' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-03T17:44:44.460 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:44.460 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [global] 2024-05-03T17:44:44.460 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout fsid = a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:44.460 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-03T17:44:44.460 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-03T17:44:44.460 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-03T17:44:44.460 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-03T17:44:44.460 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-03T17:44:44.460 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:44.461 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-03T17:44:44.461 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-03T17:44:44.461 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-05-03T17:44:44.461 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [osd] 2024-05-03T17:44:44.461 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-03T17:44:44.461 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-03T17:44:44.461 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-03T17:44:44.461 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-03T17:44:44.461 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-03T17:44:44.461 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.678+0000 7f862bb8d700 1 Processor -- start 2024-05-03T17:44:44.461 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.678+0000 7f862bb8d700 1 -- start start 2024-05-03T17:44:44.462 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.679+0000 7f862bb8d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862410bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:44.462 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.679+0000 7f862bb8d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f862407d6f0 con 0x7f86241098e0 2024-05-03T17:44:44.462 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.679+0000 7f8629929700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862410bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:44.462 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.679+0000 7f8629929700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862410bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57102/0 (socket says 172.21.15.122:57102) 2024-05-03T17:44:44.462 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.679+0000 7f8629929700 1 -- 172.21.15.122:0/1881455768 learned_addr learned my addr 172.21.15.122:0/1881455768 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:44.462 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.679+0000 7f8629929700 1 -- 172.21.15.122:0/1881455768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862410c210 con 0x7f86241098e0 2024-05-03T17:44:44.462 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.679+0000 7f8629929700 1 --2- 172.21.15.122:0/1881455768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862410bcd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f86200098a0 tx=0x7f8620009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d66be4833128aae server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:44.462 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.679+0000 7f8628927700 1 -- 172.21.15.122:0/1881455768 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8620004160 con 0x7f86241098e0 2024-05-03T17:44:44.462 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.680+0000 7f8628927700 1 -- 172.21.15.122:0/1881455768 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8620004300 con 0x7f86241098e0 2024-05-03T17:44:44.462 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.680+0000 7f8628927700 1 -- 172.21.15.122:0/1881455768 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8620011af0 con 0x7f86241098e0 2024-05-03T17:44:44.463 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.680+0000 7f862bb8d700 1 -- 172.21.15.122:0/1881455768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 msgr2=0x7f862410bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:44.463 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.680+0000 7f862bb8d700 1 --2- 172.21.15.122:0/1881455768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862410bcd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f86200098a0 tx=0x7f8620009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:44.463 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.680+0000 7f862bb8d700 1 -- 172.21.15.122:0/1881455768 shutdown_connections 2024-05-03T17:44:44.463 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.680+0000 7f862bb8d700 1 --2- 172.21.15.122:0/1881455768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862410bcd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:44.463 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.680+0000 7f862bb8d700 1 -- 172.21.15.122:0/1881455768 >> 172.21.15.122:0/1881455768 conn(0x7f86241032f0 msgr2=0x7f8624105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:44.463 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.680+0000 7f862bb8d700 1 -- 172.21.15.122:0/1881455768 shutdown_connections 2024-05-03T17:44:44.463 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.680+0000 7f862bb8d700 1 -- 172.21.15.122:0/1881455768 wait complete. 2024-05-03T17:44:44.463 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.681+0000 7f862bb8d700 1 Processor -- start 2024-05-03T17:44:44.463 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.681+0000 7f862bb8d700 1 -- start start 2024-05-03T17:44:44.463 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.681+0000 7f862bb8d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:44.463 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.681+0000 7f862bb8d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f862410d180 con 0x7f86241098e0 2024-05-03T17:44:44.464 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.681+0000 7f8629929700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:44.464 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.681+0000 7f8629929700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57106/0 (socket says 172.21.15.122:57106) 2024-05-03T17:44:44.464 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.681+0000 7f8629929700 1 -- 172.21.15.122:0/2714705037 learned_addr learned my addr 172.21.15.122:0/2714705037 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:44.464 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.681+0000 7f8629929700 1 -- 172.21.15.122:0/2714705037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862419d280 con 0x7f86241098e0 2024-05-03T17:44:44.464 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.681+0000 7f8629929700 1 --2- 172.21.15.122:0/2714705037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862419cd40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8620009fb0 tx=0x7f8620011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:44.464 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.682+0000 7f861affd700 1 -- 172.21.15.122:0/2714705037 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86200036a0 con 0x7f86241098e0 2024-05-03T17:44:44.464 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.682+0000 7f861affd700 1 -- 172.21.15.122:0/2714705037 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f862001a430 con 0x7f86241098e0 2024-05-03T17:44:44.464 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.682+0000 7f862bb8d700 1 -- 172.21.15.122:0/2714705037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862419d510 con 0x7f86241098e0 2024-05-03T17:44:44.464 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.682+0000 7f861affd700 1 -- 172.21.15.122:0/2714705037 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8620019550 con 0x7f86241098e0 2024-05-03T17:44:44.464 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.682+0000 7f862bb8d700 1 -- 172.21.15.122:0/2714705037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86241a0200 con 0x7f86241098e0 2024-05-03T17:44:44.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.683+0000 7f862bb8d700 1 -- 172.21.15.122:0/2714705037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862406a5c0 con 0x7f86241098e0 2024-05-03T17:44:44.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.683+0000 7f861affd700 1 -- 172.21.15.122:0/2714705037 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f862001a8e0 con 0x7f86241098e0 2024-05-03T17:44:44.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.683+0000 7f861affd700 1 --2- 172.21.15.122:0/2714705037 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f861003e420 0x7f86100408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:44.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.683+0000 7f861affd700 1 -- 172.21.15.122:0/2714705037 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8620053790 con 0x7f86241098e0 2024-05-03T17:44:44.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.683+0000 7f8629128700 1 --2- 172.21.15.122:0/2714705037 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f861003e420 0x7f86100408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:44.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.684+0000 7f8629128700 1 --2- 172.21.15.122:0/2714705037 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f861003e420 0x7f86100408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8614006fd0 tx=0x7f8614006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:44.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.688+0000 7f861affd700 1 -- 172.21.15.122:0/2714705037 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8620008040 con 0x7f86241098e0 2024-05-03T17:44:44.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.890+0000 7f862bb8d700 1 -- 172.21.15.122:0/2714705037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f86241a03d0 con 0x7f86241098e0 2024-05-03T17:44:44.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.891+0000 7f861affd700 1 -- 172.21.15.122:0/2714705037 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f86241a03d0 con 0x7f86241098e0 2024-05-03T17:44:44.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.893+0000 7f862bb8d700 1 -- 172.21.15.122:0/2714705037 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f861003e420 msgr2=0x7f86100408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:44.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.893+0000 7f862bb8d700 1 --2- 172.21.15.122:0/2714705037 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f861003e420 0x7f86100408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8614006fd0 tx=0x7f8614006e40 comp rx=0 tx=0).stop 2024-05-03T17:44:44.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.893+0000 7f862bb8d700 1 -- 172.21.15.122:0/2714705037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 msgr2=0x7f862419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:44.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.893+0000 7f862bb8d700 1 --2- 172.21.15.122:0/2714705037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862419cd40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8620009fb0 tx=0x7f8620011940 comp rx=0 tx=0).stop 2024-05-03T17:44:44.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.893+0000 7f862bb8d700 1 -- 172.21.15.122:0/2714705037 shutdown_connections 2024-05-03T17:44:44.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.893+0000 7f862bb8d700 1 --2- 172.21.15.122:0/2714705037 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f861003e420 0x7f86100408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:44.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.893+0000 7f862bb8d700 1 --2- 172.21.15.122:0/2714705037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86241098e0 0x7f862419cd40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:44.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.893+0000 7f862bb8d700 1 -- 172.21.15.122:0/2714705037 >> 172.21.15.122:0/2714705037 conn(0x7f86241032f0 msgr2=0x7f8624103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:44.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.893+0000 7f862bb8d700 1 -- 172.21.15.122:0/2714705037 shutdown_connections 2024-05-03T17:44:44.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:43.893+0000 7f862bb8d700 1 -- 172.21.15.122:0/2714705037 wait complete. 2024-05-03T17:44:44.466 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling cephadm module... 2024-05-03T17:44:45.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:45 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1048630609' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-03T17:44:45.461 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: ignoring --setuser ceph since I am not root 2024-05-03T17:44:45.461 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: ignoring --setgroup ceph since I am not root 2024-05-03T17:44:45.461 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:45.367+0000 7fe33b762200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T17:44:45.755 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:45.556+0000 7fe33b762200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T17:44:45.755 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:45.657+0000 7fe33b762200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T17:44:45.764 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.853+0000 7fd390705700 1 Processor -- start 2024-05-03T17:44:45.764 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.853+0000 7fd390705700 1 -- start start 2024-05-03T17:44:45.764 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.853+0000 7fd390705700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd388109230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:45.764 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.853+0000 7fd390705700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd388109770 con 0x7fd388108e10 2024-05-03T17:44:45.764 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.853+0000 7fd38e4a1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd388109230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:45.764 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.853+0000 7fd38e4a1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd388109230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57110/0 (socket says 172.21.15.122:57110) 2024-05-03T17:44:45.764 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.853+0000 7fd38e4a1700 1 -- 172.21.15.122:0/3792232788 learned_addr learned my addr 172.21.15.122:0/3792232788 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:45.764 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.854+0000 7fd38e4a1700 1 -- 172.21.15.122:0/3792232788 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd388109f70 con 0x7fd388108e10 2024-05-03T17:44:45.764 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.854+0000 7fd38e4a1700 1 --2- 172.21.15.122:0/3792232788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd388109230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd3840098a0 tx=0x7fd384009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382711a23951fd5f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.854+0000 7fd38d49f700 1 -- 172.21.15.122:0/3792232788 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384004160 con 0x7fd388108e10 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.854+0000 7fd38d49f700 1 -- 172.21.15.122:0/3792232788 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd384004300 con 0x7fd388108e10 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.854+0000 7fd38d49f700 1 -- 172.21.15.122:0/3792232788 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384011af0 con 0x7fd388108e10 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.854+0000 7fd390705700 1 -- 172.21.15.122:0/3792232788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 msgr2=0x7fd388109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.854+0000 7fd390705700 1 --2- 172.21.15.122:0/3792232788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd388109230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd3840098a0 tx=0x7fd384009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.854+0000 7fd390705700 1 -- 172.21.15.122:0/3792232788 shutdown_connections 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.855+0000 7fd390705700 1 --2- 172.21.15.122:0/3792232788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd388109230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.855+0000 7fd390705700 1 -- 172.21.15.122:0/3792232788 >> 172.21.15.122:0/3792232788 conn(0x7fd3881043d0 msgr2=0x7fd3881067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.855+0000 7fd390705700 1 -- 172.21.15.122:0/3792232788 shutdown_connections 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.855+0000 7fd390705700 1 -- 172.21.15.122:0/3792232788 wait complete. 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.855+0000 7fd390705700 1 Processor -- start 2024-05-03T17:44:45.765 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.855+0000 7fd390705700 1 -- start start 2024-05-03T17:44:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.855+0000 7fd390705700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd38819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.855+0000 7fd390705700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd38810d3b0 con 0x7fd388108e10 2024-05-03T17:44:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.855+0000 7fd38e4a1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd38819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.855+0000 7fd38e4a1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd38819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57112/0 (socket says 172.21.15.122:57112) 2024-05-03T17:44:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.855+0000 7fd38e4a1700 1 -- 172.21.15.122:0/1048630609 learned_addr learned my addr 172.21.15.122:0/1048630609 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.856+0000 7fd38e4a1700 1 -- 172.21.15.122:0/1048630609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38819d260 con 0x7fd388108e10 2024-05-03T17:44:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.856+0000 7fd38e4a1700 1 --2- 172.21.15.122:0/1048630609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd38819cd20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd384009fb0 tx=0x7fd384011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.856+0000 7fd37f7fe700 1 -- 172.21.15.122:0/1048630609 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3840036a0 con 0x7fd388108e10 2024-05-03T17:44:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.856+0000 7fd37f7fe700 1 -- 172.21.15.122:0/1048630609 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd38401a430 con 0x7fd388108e10 2024-05-03T17:44:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.856+0000 7fd390705700 1 -- 172.21.15.122:0/1048630609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38819d4f0 con 0x7fd388108e10 2024-05-03T17:44:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.856+0000 7fd37f7fe700 1 -- 172.21.15.122:0/1048630609 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384019550 con 0x7fd388108e10 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.856+0000 7fd390705700 1 -- 172.21.15.122:0/1048630609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881a01d0 con 0x7fd388108e10 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.857+0000 7fd37f7fe700 1 -- 172.21.15.122:0/1048630609 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fd38401a8e0 con 0x7fd388108e10 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.857+0000 7fd390705700 1 -- 172.21.15.122:0/1048630609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38806a5c0 con 0x7fd388108e10 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.857+0000 7fd37f7fe700 1 --2- 172.21.15.122:0/1048630609 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7fd37403e420 0x7fd3740408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.857+0000 7fd37f7fe700 1 -- 172.21.15.122:0/1048630609 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd384053790 con 0x7fd388108e10 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.857+0000 7fd38dca0700 1 --2- 172.21.15.122:0/1048630609 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7fd37403e420 0x7fd3740408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.858+0000 7fd38dca0700 1 --2- 172.21.15.122:0/1048630609 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7fd37403e420 0x7fd3740408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd378006fd0 tx=0x7fd378006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:44.862+0000 7fd37f7fe700 1 -- 172.21.15.122:0/1048630609 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd384008040 con 0x7fd388108e10 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.136+0000 7fd390705700 1 -- 172.21.15.122:0/1048630609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd3881a03a0 con 0x7fd388108e10 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.188+0000 7fd37f7fe700 1 -- 172.21.15.122:0/1048630609 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd3881a03a0 con 0x7fd388108e10 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.189+0000 7fd37f7fe700 1 -- 172.21.15.122:0/1048630609 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fd384003840 con 0x7fd388108e10 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.191+0000 7fd390705700 1 -- 172.21.15.122:0/1048630609 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7fd37403e420 msgr2=0x7fd3740408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:45.767 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.191+0000 7fd390705700 1 --2- 172.21.15.122:0/1048630609 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7fd37403e420 0x7fd3740408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd378006fd0 tx=0x7fd378006e40 comp rx=0 tx=0).stop 2024-05-03T17:44:45.768 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.191+0000 7fd390705700 1 -- 172.21.15.122:0/1048630609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 msgr2=0x7fd38819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:45.768 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.191+0000 7fd390705700 1 --2- 172.21.15.122:0/1048630609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd38819cd20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd384009fb0 tx=0x7fd384011940 comp rx=0 tx=0).stop 2024-05-03T17:44:45.768 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.191+0000 7fd390705700 1 -- 172.21.15.122:0/1048630609 shutdown_connections 2024-05-03T17:44:45.768 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.191+0000 7fd390705700 1 --2- 172.21.15.122:0/1048630609 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7fd37403e420 0x7fd3740408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:45.768 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.191+0000 7fd390705700 1 --2- 172.21.15.122:0/1048630609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108e10 0x7fd38819cd20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:45.768 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.191+0000 7fd390705700 1 -- 172.21.15.122:0/1048630609 >> 172.21.15.122:0/1048630609 conn(0x7fd3881043d0 msgr2=0x7fd388104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:45.768 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.191+0000 7fd390705700 1 -- 172.21.15.122:0/1048630609 shutdown_connections 2024-05-03T17:44:45.768 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:45.191+0000 7fd390705700 1 -- 172.21.15.122:0/1048630609 wait complete. 2024-05-03T17:44:46.042 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:45.975+0000 7fe33b762200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T17:44:46.457 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:46 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1048630609' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-03T17:44:46.457 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:46 smithi122 ceph-mon[29744]: mgrmap e5: x(active, since 6s) 2024-05-03T17:44:46.457 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:46 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:46.064+0000 7fe33b762200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T17:44:47.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.254+0000 7f893e817700 1 Processor -- start 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.254+0000 7f893e817700 1 -- start start 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.254+0000 7f893e817700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893810bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.254+0000 7f893e817700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f893807d710 con 0x7f89381098b0 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.254+0000 7f8937fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893810bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.254+0000 7f8937fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893810bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57142/0 (socket says 172.21.15.122:57142) 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.254+0000 7f8937fff700 1 -- 172.21.15.122:0/3031123956 learned_addr learned my addr 172.21.15.122:0/3031123956 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:47.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.255+0000 7f8937fff700 1 -- 172.21.15.122:0/3031123956 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893810c1e0 con 0x7f89381098b0 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.255+0000 7f8937fff700 1 --2- 172.21.15.122:0/3031123956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893810bca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f892c0098a0 tx=0x7f892c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742ed4538c31da0e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.255+0000 7f8936ffd700 1 -- 172.21.15.122:0/3031123956 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f892c004160 con 0x7f89381098b0 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.255+0000 7f8936ffd700 1 -- 172.21.15.122:0/3031123956 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f892c004300 con 0x7f89381098b0 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.255+0000 7f8936ffd700 1 -- 172.21.15.122:0/3031123956 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f892c011af0 con 0x7f89381098b0 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.256+0000 7f893e817700 1 -- 172.21.15.122:0/3031123956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 msgr2=0x7f893810bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.256+0000 7f893e817700 1 --2- 172.21.15.122:0/3031123956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893810bca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f892c0098a0 tx=0x7f892c009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.256+0000 7f893e817700 1 -- 172.21.15.122:0/3031123956 shutdown_connections 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.256+0000 7f893e817700 1 --2- 172.21.15.122:0/3031123956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893810bca0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.256+0000 7f893e817700 1 -- 172.21.15.122:0/3031123956 >> 172.21.15.122:0/3031123956 conn(0x7f89381032c0 msgr2=0x7f89381056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.256+0000 7f893e817700 1 -- 172.21.15.122:0/3031123956 shutdown_connections 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.256+0000 7f893e817700 1 -- 172.21.15.122:0/3031123956 wait complete. 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.257+0000 7f893e817700 1 Processor -- start 2024-05-03T17:44:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.257+0000 7f893e817700 1 -- start start 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.257+0000 7f893e817700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.257+0000 7f893e817700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f893810d1a0 con 0x7f89381098b0 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.257+0000 7f8937fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.257+0000 7f8937fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57154/0 (socket says 172.21.15.122:57154) 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.257+0000 7f8937fff700 1 -- 172.21.15.122:0/256941606 learned_addr learned my addr 172.21.15.122:0/256941606 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.257+0000 7f8937fff700 1 -- 172.21.15.122:0/256941606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893819d2e0 con 0x7f89381098b0 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.257+0000 7f8937fff700 1 --2- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893819cda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f892c011ac0 tx=0x7f892c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.258+0000 7f89357fa700 1 -- 172.21.15.122:0/256941606 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f892c003970 con 0x7f89381098b0 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.258+0000 7f89357fa700 1 -- 172.21.15.122:0/256941606 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f892c003b10 con 0x7f89381098b0 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.258+0000 7f893e817700 1 -- 172.21.15.122:0/256941606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893819d570 con 0x7f89381098b0 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.258+0000 7f893e817700 1 -- 172.21.15.122:0/256941606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89381a0260 con 0x7f89381098b0 2024-05-03T17:44:47.174 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.258+0000 7f89357fa700 1 -- 172.21.15.122:0/256941606 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f892c024610 con 0x7f89381098b0 2024-05-03T17:44:47.175 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.259+0000 7f89357fa700 1 -- 172.21.15.122:0/256941606 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f892c024860 con 0x7f89381098b0 2024-05-03T17:44:47.175 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.259+0000 7f893e817700 1 -- 172.21.15.122:0/256941606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893806a5c0 con 0x7f89381098b0 2024-05-03T17:44:47.175 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.259+0000 7f89357fa700 1 --2- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f892003e4c0 0x7f8920040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:47.175 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.259+0000 7f89357fa700 1 -- 172.21.15.122:0/256941606 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f892c053810 con 0x7f89381098b0 2024-05-03T17:44:47.175 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.259+0000 7f89377fe700 1 -- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f892003e4c0 msgr2=0x7f8920040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/80220221 2024-05-03T17:44:47.175 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.259+0000 7f89377fe700 1 --2- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f892003e4c0 0x7f8920040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T17:44:47.175 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.264+0000 7f89357fa700 1 -- 172.21.15.122:0/256941606 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f892c02c350 con 0x7f89381098b0 2024-05-03T17:44:47.175 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.460+0000 7f89377fe700 1 -- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f892003e4c0 msgr2=0x7f8920040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/80220221 2024-05-03T17:44:47.175 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.460+0000 7f89377fe700 1 --2- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f892003e4c0 0x7f8920040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T17:44:47.175 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.570+0000 7f893e817700 1 -- 172.21.15.122:0/256941606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f89381a0430 con 0x7f89381098b0 2024-05-03T17:44:47.175 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.570+0000 7f89357fa700 1 -- 172.21.15.122:0/256941606 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f89381a0430 con 0x7f89381098b0 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.572+0000 7f893e817700 1 -- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f892003e4c0 msgr2=0x7f8920040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.572+0000 7f893e817700 1 --2- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f892003e4c0 0x7f8920040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.572+0000 7f893e817700 1 -- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 msgr2=0x7f893819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.572+0000 7f893e817700 1 --2- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893819cda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f892c011ac0 tx=0x7f892c011ba0 comp rx=0 tx=0).stop 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.572+0000 7f893e817700 1 -- 172.21.15.122:0/256941606 shutdown_connections 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.572+0000 7f893e817700 1 --2- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f892003e4c0 0x7f8920040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.572+0000 7f893e817700 1 --2- 172.21.15.122:0/256941606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89381098b0 0x7f893819cda0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.572+0000 7f893e817700 1 -- 172.21.15.122:0/256941606 >> 172.21.15.122:0/256941606 conn(0x7f89381032c0 msgr2=0x7f8938103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.573+0000 7f893e817700 1 -- 172.21.15.122:0/256941606 shutdown_connections 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:46.573+0000 7f893e817700 1 -- 172.21.15.122:0/256941606 wait complete. 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for the mgr to restart... 2024-05-03T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr epoch 5... 2024-05-03T17:44:47.412 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:47 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/256941606' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-03T17:44:47.413 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:47 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:47.268+0000 7fe33b762200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T17:44:47.710 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:47 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:47.444+0000 7fe33b762200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T17:44:47.711 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:47 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:47.619+0000 7fe33b762200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T17:44:48.198 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:47 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:47.909+0000 7fe33b762200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T17:44:48.199 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:48 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:48.001+0000 7fe33b762200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T17:44:48.199 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:48 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:48.202+0000 7fe33b762200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T17:44:48.460 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:48 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:48.394+0000 7fe33b762200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T17:44:49.460 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:48 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:48.989+0000 7fe33b762200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T17:44:49.462 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:49 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:49.208+0000 7fe33b762200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T17:44:49.960 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:49 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:49.630+0000 7fe33b762200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T17:44:49.960 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:49 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:49.727+0000 7fe33b762200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T17:44:50.442 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:50 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:50.093+0000 7fe33b762200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T17:44:50.442 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:50 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:50.446+0000 7fe33b762200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T17:44:50.710 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:50 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:50.539+0000 7fe33b762200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T17:44:51.210 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:50 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:50.783+0000 7fe33b762200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T17:44:51.211 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:50 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:50.914+0000 7fe33b762200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T17:44:51.711 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:51 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:51.216+0000 7fe33b762200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T17:44:51.711 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:51 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:51.331+0000 7fe33b762200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T17:44:51.711 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:51 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:51.422+0000 7fe33b762200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T17:44:52.290 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:52 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:52.022+0000 7fe33b762200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T17:44:52.290 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:52 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:44:52.241+0000 7fe33b762200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T17:44:52.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: Active manager daemon x restarted 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: Activating manager daemon x 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: osdmap e2: 0 total, 0 up, 0 in 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: mgrmap e6: x(active, starting, since 0.00369021s) 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: Manager daemon x is now available 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:44:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:44:52.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T17:44:52.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:52 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T17:44:53.845 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-05-03T17:44:53.845 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-03T17:44:53.845 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-03T17:44:53.845 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-05-03T17:44:53.845 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.627+0000 7f56fd403700 1 Processor -- start 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.627+0000 7f56fd403700 1 -- start start 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.627+0000 7f56fd403700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f810bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.627+0000 7f56fd403700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56f807ce40 con 0x7f56f8109730 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.627+0000 7f56f6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f810bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.627+0000 7f56f6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f810bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57158/0 (socket says 172.21.15.122:57158) 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.627+0000 7f56f6ffd700 1 -- 172.21.15.122:0/3979590021 learned_addr learned my addr 172.21.15.122:0/3979590021 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.628+0000 7f56f6ffd700 1 -- 172.21.15.122:0/3979590021 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f810c060 con 0x7f56f8109730 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.628+0000 7f56f6ffd700 1 --2- 172.21.15.122:0/3979590021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f810bb20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0098a0 tx=0x7f56ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f637f6cd4f7c4e99 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.628+0000 7f56f5ffb700 1 -- 172.21.15.122:0/3979590021 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ec004160 con 0x7f56f8109730 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.628+0000 7f56f5ffb700 1 -- 172.21.15.122:0/3979590021 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f56ec004300 con 0x7f56f8109730 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.628+0000 7f56f5ffb700 1 -- 172.21.15.122:0/3979590021 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ec011af0 con 0x7f56f8109730 2024-05-03T17:44:53.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.628+0000 7f56fd403700 1 -- 172.21.15.122:0/3979590021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 msgr2=0x7f56f810bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.628+0000 7f56fd403700 1 --2- 172.21.15.122:0/3979590021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f810bb20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0098a0 tx=0x7f56ec009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.628+0000 7f56fd403700 1 -- 172.21.15.122:0/3979590021 shutdown_connections 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.628+0000 7f56fd403700 1 --2- 172.21.15.122:0/3979590021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f810bb20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.628+0000 7f56fd403700 1 -- 172.21.15.122:0/3979590021 >> 172.21.15.122:0/3979590021 conn(0x7f56f8103100 msgr2=0x7f56f8105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.628+0000 7f56fd403700 1 -- 172.21.15.122:0/3979590021 shutdown_connections 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.629+0000 7f56fd403700 1 -- 172.21.15.122:0/3979590021 wait complete. 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.629+0000 7f56fd403700 1 Processor -- start 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.629+0000 7f56fd403700 1 -- start start 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.629+0000 7f56fd403700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f8108fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.629+0000 7f56fd403700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56f810cfc0 con 0x7f56f8109730 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.629+0000 7f56f6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f8108fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.629+0000 7f56f6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f8108fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57162/0 (socket says 172.21.15.122:57162) 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.630+0000 7f56f6ffd700 1 -- 172.21.15.122:0/4149279638 learned_addr learned my addr 172.21.15.122:0/4149279638 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:53.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.630+0000 7f56f6ffd700 1 -- 172.21.15.122:0/4149279638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f81094f0 con 0x7f56f8109730 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.630+0000 7f56f6ffd700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f8108fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009fb0 tx=0x7f56ec011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.630+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ec0036a0 con 0x7f56f8109730 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.630+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f56ec01a430 con 0x7f56f8109730 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.630+0000 7f56fd403700 1 -- 172.21.15.122:0/4149279638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f81076e0 con 0x7f56f8109730 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.630+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ec019550 con 0x7f56f8109730 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.630+0000 7f56fd403700 1 -- 172.21.15.122:0/4149279638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f8107c20 con 0x7f56f8109730 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.631+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f56ec019850 con 0x7f56f8109730 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.631+0000 7f56dffff700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 0x7f56e00408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.631+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 --> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f56e0040ff0 con 0x7f56e003e420 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.631+0000 7f56f67fc700 1 -- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 msgr2=0x7f56e00408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/80220221 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.631+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f56ec0538b0 con 0x7f56f8109730 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.631+0000 7f56f67fc700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 0x7f56e00408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T17:44:53.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.832+0000 7f56f67fc700 1 -- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 msgr2=0x7f56e00408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/80220221 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:47.832+0000 7f56f67fc700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 0x7f56e00408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:48.233+0000 7f56f67fc700 1 -- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 msgr2=0x7f56e00408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/80220221 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:48.233+0000 7f56f67fc700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 0x7f56e00408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:49.034+0000 7f56f67fc700 1 -- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 msgr2=0x7f56e00408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/80220221 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:49.034+0000 7f56f67fc700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 0x7f56e00408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:50.635+0000 7f56f67fc700 1 -- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 msgr2=0x7f56e00408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/80220221 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:50.635+0000 7f56f67fc700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 0x7f56e00408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:52.245+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f56ec019b90 con 0x7f56f8109730 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:52.245+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 msgr2=0x7f56e00408e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:52.245+0000 7f56dffff700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 0x7f56e00408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.248+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f56ec0558c0 con 0x7f56f8109730 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.248+0000 7f56dffff700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f56e0041d10 0x7f56e0044100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.248+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 --> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f56e0040ff0 con 0x7f56e0041d10 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.249+0000 7f56f67fc700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f56e0041d10 0x7f56e0044100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:53.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.250+0000 7f56f67fc700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f56e0041d10 0x7f56e0044100 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f56e8003a10 tx=0x7f56e8009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.250+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 <== mgr.14120 v2:172.21.15.122:6800/1659741627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f56e0040ff0 con 0x7f56e0041d10 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.258+0000 7f56fd403700 1 -- 172.21.15.122:0/4149279638 --> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f56f806a5c0 con 0x7f56e0041d10 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.258+0000 7f56dffff700 1 -- 172.21.15.122:0/4149279638 <== mgr.14120 v2:172.21.15.122:6800/1659741627 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f56f806a5c0 con 0x7f56e0041d10 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.259+0000 7f56fd403700 1 -- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f56e0041d10 msgr2=0x7f56e0044100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.259+0000 7f56fd403700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f56e0041d10 0x7f56e0044100 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f56e8003a10 tx=0x7f56e8009290 comp rx=0 tx=0).stop 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.259+0000 7f56fd403700 1 -- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 msgr2=0x7f56f8108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.259+0000 7f56fd403700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f8108fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009fb0 tx=0x7f56ec011940 comp rx=0 tx=0).stop 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.259+0000 7f56fd403700 1 -- 172.21.15.122:0/4149279638 shutdown_connections 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.259+0000 7f56fd403700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f56e0041d10 0x7f56e0044100 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.259+0000 7f56fd403700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f8109730 0x7f56f8108fb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.259+0000 7f56fd403700 1 --2- 172.21.15.122:0/4149279638 >> [v2:172.21.15.122:6800/80220221,v1:172.21.15.122:6801/80220221] conn(0x7f56e003e420 0x7f56e00408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.259+0000 7f56fd403700 1 -- 172.21.15.122:0/4149279638 >> 172.21.15.122:0/4149279638 conn(0x7f56f8103100 msgr2=0x7f56f8103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:53.850 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.259+0000 7f56fd403700 1 -- 172.21.15.122:0/4149279638 shutdown_connections 2024-05-03T17:44:53.851 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:53.259+0000 7f56fd403700 1 -- 172.21.15.122:0/4149279638 wait complete. 2024-05-03T17:44:53.851 INFO:teuthology.orchestra.run.smithi122.stdout:mgr epoch 5 is available 2024-05-03T17:44:53.851 INFO:teuthology.orchestra.run.smithi122.stdout:Setting orchestrator backend to cephadm... 2024-05-03T17:44:54.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:54 smithi122 ceph-mon[29744]: Found migration_current of "None". Setting to last migration. 2024-05-03T17:44:54.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:54 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:44:54.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:54 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:44:54.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:54 smithi122 ceph-mon[29744]: mgrmap e7: x(active, since 1.00761s) 2024-05-03T17:44:54.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:54 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:44:55.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.242+0000 7f9193238700 1 Processor -- start 2024-05-03T17:44:55.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.242+0000 7f9193238700 1 -- start start 2024-05-03T17:44:55.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.242+0000 7f9193238700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c109230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:55.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.242+0000 7f9193238700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f918c109770 con 0x7f918c108e10 2024-05-03T17:44:55.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.242+0000 7f9190fd4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c109230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:55.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.242+0000 7f9190fd4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c109230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58660/0 (socket says 172.21.15.122:58660) 2024-05-03T17:44:55.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.242+0000 7f9190fd4700 1 -- 172.21.15.122:0/1403723850 learned_addr learned my addr 172.21.15.122:0/1403723850 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:55.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.242+0000 7f9190fd4700 1 -- 172.21.15.122:0/1403723850 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c109f70 con 0x7f918c108e10 2024-05-03T17:44:55.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.242+0000 7f9190fd4700 1 --2- 172.21.15.122:0/1403723850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c109230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91800098a0 tx=0x7f9180009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdbd8ae556fa8ed server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:55.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.242+0000 7f918b7fe700 1 -- 172.21.15.122:0/1403723850 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9180004160 con 0x7f918c108e10 2024-05-03T17:44:55.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.243+0000 7f918b7fe700 1 -- 172.21.15.122:0/1403723850 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9180004300 con 0x7f918c108e10 2024-05-03T17:44:55.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.243+0000 7f918b7fe700 1 -- 172.21.15.122:0/1403723850 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9180011af0 con 0x7f918c108e10 2024-05-03T17:44:55.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.243+0000 7f9193238700 1 -- 172.21.15.122:0/1403723850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 msgr2=0x7f918c109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:55.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.243+0000 7f9193238700 1 --2- 172.21.15.122:0/1403723850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c109230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91800098a0 tx=0x7f9180009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:55.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.243+0000 7f9193238700 1 -- 172.21.15.122:0/1403723850 shutdown_connections 2024-05-03T17:44:55.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.243+0000 7f9193238700 1 --2- 172.21.15.122:0/1403723850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c109230 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:55.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.243+0000 7f9193238700 1 -- 172.21.15.122:0/1403723850 >> 172.21.15.122:0/1403723850 conn(0x7f918c1043d0 msgr2=0x7f918c1067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:55.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.243+0000 7f9193238700 1 -- 172.21.15.122:0/1403723850 shutdown_connections 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.243+0000 7f9193238700 1 -- 172.21.15.122:0/1403723850 wait complete. 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.244+0000 7f9193238700 1 Processor -- start 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.244+0000 7f9193238700 1 -- start start 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.244+0000 7f9193238700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.244+0000 7f9193238700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f918c10d3b0 con 0x7f918c108e10 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.244+0000 7f9190fd4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.244+0000 7f9190fd4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58668/0 (socket says 172.21.15.122:58668) 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.244+0000 7f9190fd4700 1 -- 172.21.15.122:0/1375342908 learned_addr learned my addr 172.21.15.122:0/1375342908 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.244+0000 7f9190fd4700 1 -- 172.21.15.122:0/1375342908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c19d260 con 0x7f918c108e10 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.244+0000 7f9190fd4700 1 --2- 172.21.15.122:0/1375342908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c19cd20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9180009fb0 tx=0x7f9180011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.244+0000 7f9189ffb700 1 -- 172.21.15.122:0/1375342908 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91800036a0 con 0x7f918c108e10 2024-05-03T17:44:55.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.245+0000 7f9189ffb700 1 -- 172.21.15.122:0/1375342908 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f918001a430 con 0x7f918c108e10 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.245+0000 7f9193238700 1 -- 172.21.15.122:0/1375342908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c19d4f0 con 0x7f918c108e10 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.245+0000 7f9189ffb700 1 -- 172.21.15.122:0/1375342908 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9180019550 con 0x7f918c108e10 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.245+0000 7f9193238700 1 -- 172.21.15.122:0/1375342908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c1a01d0 con 0x7f918c108e10 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.246+0000 7f9189ffb700 1 -- 172.21.15.122:0/1375342908 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f9180019800 con 0x7f918c108e10 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.246+0000 7f9193238700 1 -- 172.21.15.122:0/1375342908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c06a5c0 con 0x7f918c108e10 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.246+0000 7f9189ffb700 1 --2- 172.21.15.122:0/1375342908 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f917c03e340 0x7f917c040800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.246+0000 7f9189ffb700 1 -- 172.21.15.122:0/1375342908 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9180053680 con 0x7f918c108e10 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.246+0000 7f918bfff700 1 --2- 172.21.15.122:0/1375342908 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f917c03e340 0x7f917c040800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.246+0000 7f918bfff700 1 --2- 172.21.15.122:0/1375342908 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f917c03e340 0x7f917c040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9178006fd0 tx=0x7f9178006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.251+0000 7f9189ffb700 1 -- 172.21.15.122:0/1375342908 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9180017e90 con 0x7f918c108e10 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.472+0000 7f9193238700 1 -- 172.21.15.122:0/1375342908 --> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f918c105580 con 0x7f917c03e340 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.478+0000 7f9189ffb700 1 -- 172.21.15.122:0/1375342908 <== mgr.14120 v2:172.21.15.122:6800/1659741627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f918c105580 con 0x7f917c03e340 2024-05-03T17:44:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.480+0000 7f9193238700 1 -- 172.21.15.122:0/1375342908 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f917c03e340 msgr2=0x7f917c040800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:55.005 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.480+0000 7f9193238700 1 --2- 172.21.15.122:0/1375342908 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f917c03e340 0x7f917c040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9178006fd0 tx=0x7f9178006e40 comp rx=0 tx=0).stop 2024-05-03T17:44:55.005 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.480+0000 7f9193238700 1 -- 172.21.15.122:0/1375342908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 msgr2=0x7f918c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:55.005 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.480+0000 7f9193238700 1 --2- 172.21.15.122:0/1375342908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c19cd20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9180009fb0 tx=0x7f9180011940 comp rx=0 tx=0).stop 2024-05-03T17:44:55.005 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.480+0000 7f9193238700 1 -- 172.21.15.122:0/1375342908 shutdown_connections 2024-05-03T17:44:55.005 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.480+0000 7f9193238700 1 --2- 172.21.15.122:0/1375342908 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f917c03e340 0x7f917c040800 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:55.005 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.480+0000 7f9193238700 1 --2- 172.21.15.122:0/1375342908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f918c108e10 0x7f918c19cd20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:55.005 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.480+0000 7f9193238700 1 -- 172.21.15.122:0/1375342908 >> 172.21.15.122:0/1375342908 conn(0x7f918c1043d0 msgr2=0x7f918c104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:55.005 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.480+0000 7f9193238700 1 -- 172.21.15.122:0/1375342908 shutdown_connections 2024-05-03T17:44:55.005 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:54.480+0000 7f9193238700 1 -- 172.21.15.122:0/1375342908 wait complete. 2024-05-03T17:44:55.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:55 smithi122 ceph-mon[29744]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-03T17:44:55.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:55 smithi122 ceph-mon[29744]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-03T17:44:55.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:55 smithi122 ceph-mon[29744]: [03/May/2024:17:44:53] ENGINE Bus STARTING 2024-05-03T17:44:55.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:55 smithi122 ceph-mon[29744]: [03/May/2024:17:44:53] ENGINE Serving on http://172.21.15.122:8765 2024-05-03T17:44:55.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:55 smithi122 ceph-mon[29744]: [03/May/2024:17:44:53] ENGINE Serving on https://172.21.15.122:7150 2024-05-03T17:44:55.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:55 smithi122 ceph-mon[29744]: [03/May/2024:17:44:53] ENGINE Bus STARTED 2024-05-03T17:44:55.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:55 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:44:55.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:55 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:44:56.257 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:56 smithi122 ceph-mon[29744]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:44:56.257 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:56 smithi122 ceph-mon[29744]: mgrmap e8: x(active, since 2s) 2024-05-03T17:44:56.265 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-03T17:44:56.265 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.445+0000 7f9d28111700 1 Processor -- start 2024-05-03T17:44:56.265 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.445+0000 7f9d28111700 1 -- start start 2024-05-03T17:44:56.265 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.445+0000 7f9d28111700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d20109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:56.266 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.445+0000 7f9d28111700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d201097c0 con 0x7f9d20108e60 2024-05-03T17:44:56.266 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.445+0000 7f9d25ead700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d20109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:56.266 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.445+0000 7f9d25ead700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d20109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58678/0 (socket says 172.21.15.122:58678) 2024-05-03T17:44:56.266 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.445+0000 7f9d25ead700 1 -- 172.21.15.122:0/2203801995 learned_addr learned my addr 172.21.15.122:0/2203801995 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:56.266 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.445+0000 7f9d25ead700 1 -- 172.21.15.122:0/2203801995 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d20109f70 con 0x7f9d20108e60 2024-05-03T17:44:56.266 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.446+0000 7f9d25ead700 1 --2- 172.21.15.122:0/2203801995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d20109280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0098a0 tx=0x7f9d1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b44231f3faca9d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:56.266 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.446+0000 7f9d24eab700 1 -- 172.21.15.122:0/2203801995 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d1c004160 con 0x7f9d20108e60 2024-05-03T17:44:56.266 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.446+0000 7f9d24eab700 1 -- 172.21.15.122:0/2203801995 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9d1c004300 con 0x7f9d20108e60 2024-05-03T17:44:56.266 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.446+0000 7f9d24eab700 1 -- 172.21.15.122:0/2203801995 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d1c011af0 con 0x7f9d20108e60 2024-05-03T17:44:56.266 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.446+0000 7f9d28111700 1 -- 172.21.15.122:0/2203801995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 msgr2=0x7f9d20109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:56.266 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.446+0000 7f9d28111700 1 --2- 172.21.15.122:0/2203801995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d20109280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0098a0 tx=0x7f9d1c009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:56.267 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.446+0000 7f9d28111700 1 -- 172.21.15.122:0/2203801995 shutdown_connections 2024-05-03T17:44:56.267 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.446+0000 7f9d28111700 1 --2- 172.21.15.122:0/2203801995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d20109280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:56.267 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.446+0000 7f9d28111700 1 -- 172.21.15.122:0/2203801995 >> 172.21.15.122:0/2203801995 conn(0x7f9d20104440 msgr2=0x7f9d20106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:56.267 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.446+0000 7f9d28111700 1 -- 172.21.15.122:0/2203801995 shutdown_connections 2024-05-03T17:44:56.267 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.447+0000 7f9d28111700 1 -- 172.21.15.122:0/2203801995 wait complete. 2024-05-03T17:44:56.267 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.447+0000 7f9d28111700 1 Processor -- start 2024-05-03T17:44:56.267 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.447+0000 7f9d28111700 1 -- start start 2024-05-03T17:44:56.267 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.447+0000 7f9d28111700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d2019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:56.267 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.447+0000 7f9d28111700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2010d3b0 con 0x7f9d20108e60 2024-05-03T17:44:56.267 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.447+0000 7f9d25ead700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d2019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:56.267 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.447+0000 7f9d25ead700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d2019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58692/0 (socket says 172.21.15.122:58692) 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.447+0000 7f9d25ead700 1 -- 172.21.15.122:0/1585232835 learned_addr learned my addr 172.21.15.122:0/1585232835 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.448+0000 7f9d25ead700 1 -- 172.21.15.122:0/1585232835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2019d240 con 0x7f9d20108e60 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.448+0000 7f9d25ead700 1 --2- 172.21.15.122:0/1585232835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d2019cd00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c011ac0 tx=0x7f9d1c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.448+0000 7f9d16ffd700 1 -- 172.21.15.122:0/1585232835 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d1c003970 con 0x7f9d20108e60 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.448+0000 7f9d28111700 1 -- 172.21.15.122:0/1585232835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2019d4d0 con 0x7f9d20108e60 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.448+0000 7f9d16ffd700 1 -- 172.21.15.122:0/1585232835 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9d1c003b10 con 0x7f9d20108e60 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.448+0000 7f9d16ffd700 1 -- 172.21.15.122:0/1585232835 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d1c024610 con 0x7f9d20108e60 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.448+0000 7f9d28111700 1 -- 172.21.15.122:0/1585232835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d201a01b0 con 0x7f9d20108e60 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.449+0000 7f9d16ffd700 1 -- 172.21.15.122:0/1585232835 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f9d1c01a430 con 0x7f9d20108e60 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.449+0000 7f9d28111700 1 -- 172.21.15.122:0/1585232835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2006a5c0 con 0x7f9d20108e60 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.449+0000 7f9d16ffd700 1 --2- 172.21.15.122:0/1585232835 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f9d0c03e4c0 0x7f9d0c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:56.268 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.449+0000 7f9d16ffd700 1 -- 172.21.15.122:0/1585232835 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9d1c053800 con 0x7f9d20108e60 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.449+0000 7f9d256ac700 1 --2- 172.21.15.122:0/1585232835 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f9d0c03e4c0 0x7f9d0c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.450+0000 7f9d256ac700 1 --2- 172.21.15.122:0/1585232835 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f9d0c03e4c0 0x7f9d0c040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d10006fd0 tx=0x7f9d10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.454+0000 7f9d16ffd700 1 -- 172.21.15.122:0/1585232835 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9d1c00b3e0 con 0x7f9d20108e60 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.663+0000 7f9d28111700 1 -- 172.21.15.122:0/1585232835 --> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9d201055a0 con 0x7f9d0c03e4c0 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.663+0000 7f9d16ffd700 1 -- 172.21.15.122:0/1585232835 <== mgr.14120 v2:172.21.15.122:6800/1659741627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9d201055a0 con 0x7f9d0c03e4c0 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.665+0000 7f9d28111700 1 -- 172.21.15.122:0/1585232835 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f9d0c03e4c0 msgr2=0x7f9d0c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.665+0000 7f9d28111700 1 --2- 172.21.15.122:0/1585232835 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f9d0c03e4c0 0x7f9d0c040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d10006fd0 tx=0x7f9d10006e40 comp rx=0 tx=0).stop 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.665+0000 7f9d28111700 1 -- 172.21.15.122:0/1585232835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 msgr2=0x7f9d2019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.665+0000 7f9d28111700 1 --2- 172.21.15.122:0/1585232835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d2019cd00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c011ac0 tx=0x7f9d1c011ba0 comp rx=0 tx=0).stop 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.665+0000 7f9d28111700 1 -- 172.21.15.122:0/1585232835 shutdown_connections 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.665+0000 7f9d28111700 1 --2- 172.21.15.122:0/1585232835 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f9d0c03e4c0 0x7f9d0c040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:56.269 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.665+0000 7f9d28111700 1 --2- 172.21.15.122:0/1585232835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d20108e60 0x7f9d2019cd00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:56.270 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.666+0000 7f9d28111700 1 -- 172.21.15.122:0/1585232835 >> 172.21.15.122:0/1585232835 conn(0x7f9d20104440 msgr2=0x7f9d20104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:56.270 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.666+0000 7f9d28111700 1 -- 172.21.15.122:0/1585232835 shutdown_connections 2024-05-03T17:44:56.270 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:55.666+0000 7f9d28111700 1 -- 172.21.15.122:0/1585232835 wait complete. 2024-05-03T17:44:56.270 INFO:teuthology.orchestra.run.smithi122.stdout:Generating ssh key... 2024-05-03T17:44:57.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:57 smithi122 ceph-mon[29744]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:44:57.211 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: Generating public/private rsa key pair. 2024-05-03T17:44:57.211 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: Your identification has been saved in /tmp/tmpozn05c2v/key. 2024-05-03T17:44:57.211 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: Your public key has been saved in /tmp/tmpozn05c2v/key.pub. 2024-05-03T17:44:57.211 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: The key fingerprint is: 2024-05-03T17:44:57.211 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: SHA256:DIuXeIyHfMRo4/dvU0BePTwX1r9C1eRXcvBrQEW+zjo ceph-a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: The key's randomart image is: 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: +---[RSA 3072]----+ 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: | +=BB| 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: | o . o B*=| 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: | + + o . ..==| 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: | + O = o .. =| 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: | B X S .. +.| 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: | * . ..+. | 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: | . . .o | 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: | .o E. | 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: | ... .. | 2024-05-03T17:44:57.212 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:44:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: +----[SHA256]-----+ 2024-05-03T17:44:57.623 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.709+0000 7ff5f36e3700 1 Processor -- start 2024-05-03T17:44:57.623 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.709+0000 7ff5f36e3700 1 -- start start 2024-05-03T17:44:57.623 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.709+0000 7ff5f36e3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec109230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:57.623 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.709+0000 7ff5f36e3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec109770 con 0x7ff5ec108e10 2024-05-03T17:44:57.623 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.710+0000 7ff5f147f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec109230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:57.623 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.710+0000 7ff5f147f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec109230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58698/0 (socket says 172.21.15.122:58698) 2024-05-03T17:44:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.710+0000 7ff5f147f700 1 -- 172.21.15.122:0/2739077860 learned_addr learned my addr 172.21.15.122:0/2739077860 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.710+0000 7ff5f147f700 1 -- 172.21.15.122:0/2739077860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec109f70 con 0x7ff5ec108e10 2024-05-03T17:44:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.710+0000 7ff5f147f700 1 --2- 172.21.15.122:0/2739077860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec109230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009e80 tx=0x7ff5e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de62aa7cc5ea9a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.710+0000 7ff5e3fff700 1 -- 172.21.15.122:0/2739077860 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e8017070 con 0x7ff5ec108e10 2024-05-03T17:44:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.710+0000 7ff5e3fff700 1 -- 172.21.15.122:0/2739077860 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff5e800b8c0 con 0x7ff5ec108e10 2024-05-03T17:44:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.710+0000 7ff5e3fff700 1 -- 172.21.15.122:0/2739077860 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e8004490 con 0x7ff5ec108e10 2024-05-03T17:44:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.711+0000 7ff5f36e3700 1 -- 172.21.15.122:0/2739077860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 msgr2=0x7ff5ec109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.711+0000 7ff5f36e3700 1 --2- 172.21.15.122:0/2739077860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec109230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009e80 tx=0x7ff5e800b330 comp rx=0 tx=0).stop 2024-05-03T17:44:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.711+0000 7ff5f36e3700 1 -- 172.21.15.122:0/2739077860 shutdown_connections 2024-05-03T17:44:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.711+0000 7ff5f36e3700 1 --2- 172.21.15.122:0/2739077860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec109230 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.711+0000 7ff5f36e3700 1 -- 172.21.15.122:0/2739077860 >> 172.21.15.122:0/2739077860 conn(0x7ff5ec1043d0 msgr2=0x7ff5ec1067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:57.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.711+0000 7ff5f36e3700 1 -- 172.21.15.122:0/2739077860 shutdown_connections 2024-05-03T17:44:57.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.711+0000 7ff5f36e3700 1 -- 172.21.15.122:0/2739077860 wait complete. 2024-05-03T17:44:57.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.711+0000 7ff5f36e3700 1 Processor -- start 2024-05-03T17:44:57.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.711+0000 7ff5f36e3700 1 -- start start 2024-05-03T17:44:57.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5f36e3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:57.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5f36e3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec10d3b0 con 0x7ff5ec108e10 2024-05-03T17:44:57.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5f147f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:57.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5f147f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58710/0 (socket says 172.21.15.122:58710) 2024-05-03T17:44:57.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5f147f700 1 -- 172.21.15.122:0/1360097738 learned_addr learned my addr 172.21.15.122:0/1360097738 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:57.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5f147f700 1 -- 172.21.15.122:0/1360097738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec19d200 con 0x7ff5ec108e10 2024-05-03T17:44:57.625 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5f147f700 1 --2- 172.21.15.122:0/1360097738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec19ccc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800b7b0 tx=0x7ff5e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:57.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5e27fc700 1 -- 172.21.15.122:0/1360097738 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e8020020 con 0x7ff5ec108e10 2024-05-03T17:44:57.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5f36e3700 1 -- 172.21.15.122:0/1360097738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec19d490 con 0x7ff5ec108e10 2024-05-03T17:44:57.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5e27fc700 1 -- 172.21.15.122:0/1360097738 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff5e8012430 con 0x7ff5ec108e10 2024-05-03T17:44:57.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5e27fc700 1 -- 172.21.15.122:0/1360097738 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e8017040 con 0x7ff5ec108e10 2024-05-03T17:44:57.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.712+0000 7ff5f36e3700 1 -- 172.21.15.122:0/1360097738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec1a0180 con 0x7ff5ec108e10 2024-05-03T17:44:57.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.713+0000 7ff5e27fc700 1 -- 172.21.15.122:0/1360097738 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ff5e8023020 con 0x7ff5ec108e10 2024-05-03T17:44:57.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.713+0000 7ff5f36e3700 1 -- 172.21.15.122:0/1360097738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ec06a5c0 con 0x7ff5ec108e10 2024-05-03T17:44:57.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.713+0000 7ff5e27fc700 1 --2- 172.21.15.122:0/1360097738 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7ff5d803e420 0x7ff5d80408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:57.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.714+0000 7ff5e27fc700 1 -- 172.21.15.122:0/1360097738 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff5e800f050 con 0x7ff5ec108e10 2024-05-03T17:44:57.626 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.714+0000 7ff5f0c7e700 1 --2- 172.21.15.122:0/1360097738 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7ff5d803e420 0x7ff5d80408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:57.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.714+0000 7ff5f0c7e700 1 --2- 172.21.15.122:0/1360097738 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7ff5d803e420 0x7ff5d80408e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc006fd0 tx=0x7ff5dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:57.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.719+0000 7ff5e27fc700 1 -- 172.21.15.122:0/1360097738 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff5e802a020 con 0x7ff5ec108e10 2024-05-03T17:44:57.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:56.925+0000 7ff5f36e3700 1 -- 172.21.15.122:0/1360097738 --> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7ff5ec1054e0 con 0x7ff5d803e420 2024-05-03T17:44:57.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:57.101+0000 7ff5e27fc700 1 -- 172.21.15.122:0/1360097738 <== mgr.14120 v2:172.21.15.122:6800/1659741627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff5ec1054e0 con 0x7ff5d803e420 2024-05-03T17:44:57.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:57.104+0000 7ff5f36e3700 1 -- 172.21.15.122:0/1360097738 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7ff5d803e420 msgr2=0x7ff5d80408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:57.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:57.104+0000 7ff5f36e3700 1 --2- 172.21.15.122:0/1360097738 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7ff5d803e420 0x7ff5d80408e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc006fd0 tx=0x7ff5dc006e40 comp rx=0 tx=0).stop 2024-05-03T17:44:57.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:57.104+0000 7ff5f36e3700 1 -- 172.21.15.122:0/1360097738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 msgr2=0x7ff5ec19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:57.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:57.104+0000 7ff5f36e3700 1 --2- 172.21.15.122:0/1360097738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec19ccc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800b7b0 tx=0x7ff5e800be20 comp rx=0 tx=0).stop 2024-05-03T17:44:57.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:57.104+0000 7ff5f36e3700 1 -- 172.21.15.122:0/1360097738 shutdown_connections 2024-05-03T17:44:57.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:57.104+0000 7ff5f36e3700 1 --2- 172.21.15.122:0/1360097738 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7ff5d803e420 0x7ff5d80408e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:57.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:57.104+0000 7ff5f36e3700 1 --2- 172.21.15.122:0/1360097738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ec108e10 0x7ff5ec19ccc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:57.628 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:57.104+0000 7ff5f36e3700 1 -- 172.21.15.122:0/1360097738 >> 172.21.15.122:0/1360097738 conn(0x7ff5ec1043d0 msgr2=0x7ff5ec104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:57.628 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:57.104+0000 7ff5f36e3700 1 -- 172.21.15.122:0/1360097738 shutdown_connections 2024-05-03T17:44:57.628 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:57.104+0000 7ff5f36e3700 1 -- 172.21.15.122:0/1360097738 wait complete. 2024-05-03T17:44:58.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:58 smithi122 ceph-mon[29744]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:44:58.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:58 smithi122 ceph-mon[29744]: Generating ssh key... 2024-05-03T17:44:58.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:58 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:44:58.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:58 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMLpBLSSoKkrRndKn/089lrnZFCLNv4yBbaog3HZOhWegepqWW2M1sbkMSu3YTqUau57wAmFUzE7IFQyWD0QkDCgLJqsaHMOYNTuLU8ToTNFL77gshDG2Qn+rU7oPqjdZF09lhtaoljTEKHymaopi/QwEBb+VsG0FfyrFIaUBedE3EL2Ibv8OFSPQe4nkltiU9eYmHbygEKRZFJhElhc53Qh2cZrbMNmcxc7iTAr3OYU/Pmra7diLth/p1iuF74IuGt+Zbh4SdlSjj99U7jQ+6QLBL8+1exU45bi6deNtLP76CuvHHM8F1Ki8g5cITlnR/CU8XerpJimuI/nE1nw8ejnmPleeDSWx1+RKtJxi0ICZ1+WnV3OqepZmt1COhWReQPWMPixDQ+YOpmY9GR0grogBaxvFSmN57INwFbL/dUW+dfEDn1+1NBRKisYjba/2t8bND9bKoVM2k/sUTI9VAa0jGmdXdmFGr4AUEbvi31/mt5vh6IOOFfCEWQwh4Yuc= ceph-a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.054+0000 7f18d860c700 1 Processor -- start 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.054+0000 7f18d860c700 1 -- start start 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.054+0000 7f18d860c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d007dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.054+0000 7f18d860c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18d007e170 con 0x7f18d007f6f0 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.055+0000 7f18d63a8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d007dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.055+0000 7f18d63a8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d007dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58714/0 (socket says 172.21.15.122:58714) 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.055+0000 7f18d63a8700 1 -- 172.21.15.122:0/34611725 learned_addr learned my addr 172.21.15.122:0/34611725 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.055+0000 7f18d63a8700 1 -- 172.21.15.122:0/34611725 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d007e2f0 con 0x7f18d007f6f0 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.055+0000 7f18d63a8700 1 --2- 172.21.15.122:0/34611725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d007dba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18c00098a0 tx=0x7f18c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8cc8d9b57d212c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.055+0000 7f18d53a6700 1 -- 172.21.15.122:0/34611725 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18c0004160 con 0x7f18d007f6f0 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.055+0000 7f18d53a6700 1 -- 172.21.15.122:0/34611725 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f18c0004300 con 0x7f18d007f6f0 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.055+0000 7f18d53a6700 1 -- 172.21.15.122:0/34611725 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18c0011af0 con 0x7f18d007f6f0 2024-05-03T17:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.056+0000 7f18d860c700 1 -- 172.21.15.122:0/34611725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 msgr2=0x7f18d007dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.056+0000 7f18d860c700 1 --2- 172.21.15.122:0/34611725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d007dba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18c00098a0 tx=0x7f18c0009bb0 comp rx=0 tx=0).stop 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.056+0000 7f18d860c700 1 -- 172.21.15.122:0/34611725 shutdown_connections 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.056+0000 7f18d860c700 1 --2- 172.21.15.122:0/34611725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d007dba0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.056+0000 7f18d860c700 1 -- 172.21.15.122:0/34611725 >> 172.21.15.122:0/34611725 conn(0x7f18d0104410 msgr2=0x7f18d0106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.056+0000 7f18d860c700 1 -- 172.21.15.122:0/34611725 shutdown_connections 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.056+0000 7f18d860c700 1 -- 172.21.15.122:0/34611725 wait complete. 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.056+0000 7f18d860c700 1 Processor -- start 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18d860c700 1 -- start start 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18d860c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18d860c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18d010f380 con 0x7f18d007f6f0 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18d63a8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18d63a8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58718/0 (socket says 172.21.15.122:58718) 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18d63a8700 1 -- 172.21.15.122:0/1857026964 learned_addr learned my addr 172.21.15.122:0/1857026964 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:44:58.847 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18d63a8700 1 -- 172.21.15.122:0/1857026964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d019f4a0 con 0x7f18d007f6f0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18d63a8700 1 --2- 172.21.15.122:0/1857026964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d019ef60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18c0011ac0 tx=0x7f18c0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18c77fe700 1 -- 172.21.15.122:0/1857026964 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18c0003970 con 0x7f18d007f6f0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18c77fe700 1 -- 172.21.15.122:0/1857026964 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f18c0003b10 con 0x7f18d007f6f0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18c77fe700 1 -- 172.21.15.122:0/1857026964 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18c0024610 con 0x7f18d007f6f0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18d860c700 1 -- 172.21.15.122:0/1857026964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d019f730 con 0x7f18d007f6f0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.057+0000 7f18d860c700 1 -- 172.21.15.122:0/1857026964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01a2420 con 0x7f18d007f6f0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.058+0000 7f18c77fe700 1 -- 172.21.15.122:0/1857026964 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f18c001a9e0 con 0x7f18d007f6f0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.058+0000 7f18d860c700 1 -- 172.21.15.122:0/1857026964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d006a5c0 con 0x7f18d007f6f0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.058+0000 7f18c77fe700 1 --2- 172.21.15.122:0/1857026964 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f18bc03e4c0 0x7f18bc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.058+0000 7f18c77fe700 1 -- 172.21.15.122:0/1857026964 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f18c0053850 con 0x7f18d007f6f0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.059+0000 7f18d5ba7700 1 --2- 172.21.15.122:0/1857026964 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f18bc03e4c0 0x7f18bc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.059+0000 7f18d5ba7700 1 --2- 172.21.15.122:0/1857026964 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f18bc03e4c0 0x7f18bc040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f18cc006fd0 tx=0x7f18cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:44:58.848 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.063+0000 7f18c77fe700 1 -- 172.21.15.122:0/1857026964 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f18c00247f0 con 0x7f18d007f6f0 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.274+0000 7f18d860c700 1 -- 172.21.15.122:0/1857026964 --> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f18d0105690 con 0x7f18bc03e4c0 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.275+0000 7f18c77fe700 1 -- 172.21.15.122:0/1857026964 <== mgr.14120 v2:172.21.15.122:6800/1659741627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f18d0105690 con 0x7f18bc03e4c0 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.277+0000 7f18d860c700 1 -- 172.21.15.122:0/1857026964 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f18bc03e4c0 msgr2=0x7f18bc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.277+0000 7f18d860c700 1 --2- 172.21.15.122:0/1857026964 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f18bc03e4c0 0x7f18bc040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f18cc006fd0 tx=0x7f18cc006e40 comp rx=0 tx=0).stop 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.277+0000 7f18d860c700 1 -- 172.21.15.122:0/1857026964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 msgr2=0x7f18d019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.277+0000 7f18d860c700 1 --2- 172.21.15.122:0/1857026964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d019ef60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18c0011ac0 tx=0x7f18c0011ba0 comp rx=0 tx=0).stop 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.277+0000 7f18d860c700 1 -- 172.21.15.122:0/1857026964 shutdown_connections 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.277+0000 7f18d860c700 1 --2- 172.21.15.122:0/1857026964 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f18bc03e4c0 0x7f18bc040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.277+0000 7f18d860c700 1 --2- 172.21.15.122:0/1857026964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d007f6f0 0x7f18d019ef60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.277+0000 7f18d860c700 1 -- 172.21.15.122:0/1857026964 >> 172.21.15.122:0/1857026964 conn(0x7f18d0104410 msgr2=0x7f18d0104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.277+0000 7f18d860c700 1 -- 172.21.15.122:0/1857026964 shutdown_connections 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:58.277+0000 7f18d860c700 1 -- 172.21.15.122:0/1857026964 wait complete. 2024-05-03T17:44:58.849 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-03T17:44:58.850 INFO:teuthology.orchestra.run.smithi122.stdout:Adding key to root@localhost authorized_keys... 2024-05-03T17:44:58.850 INFO:teuthology.orchestra.run.smithi122.stdout:Adding host smithi122... 2024-05-03T17:44:59.522 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:44:59 smithi122 ceph-mon[29744]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:45:00.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:00 smithi122 ceph-mon[29744]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi122", "addr": "172.21.15.122", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:45:00.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:00 smithi122 ceph-mon[29744]: Deploying cephadm binary to smithi122 2024-05-03T17:45:04.129 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Added host 'smithi122' with addr '172.21.15.122' 2024-05-03T17:45:04.129 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.261+0000 7fd58f2e3700 1 Processor -- start 2024-05-03T17:45:04.129 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.261+0000 7fd58f2e3700 1 -- start start 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.261+0000 7fd58f2e3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58807dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.261+0000 7fd58f2e3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd58807e190 con 0x7fd58807f760 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.261+0000 7fd58d07f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58807dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.261+0000 7fd58d07f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58807dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53546/0 (socket says 172.21.15.122:53546) 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.261+0000 7fd58d07f700 1 -- 172.21.15.122:0/4127232686 learned_addr learned my addr 172.21.15.122:0/4127232686 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.261+0000 7fd58d07f700 1 -- 172.21.15.122:0/4127232686 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58807e310 con 0x7fd58807f760 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.262+0000 7fd58d07f700 1 --2- 172.21.15.122:0/4127232686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58807dbc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd5840098a0 tx=0x7fd584009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeae1d8f9777ba59 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.262+0000 7fd57ffff700 1 -- 172.21.15.122:0/4127232686 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd584004160 con 0x7fd58807f760 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.262+0000 7fd57ffff700 1 -- 172.21.15.122:0/4127232686 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd584004300 con 0x7fd58807f760 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.262+0000 7fd57ffff700 1 -- 172.21.15.122:0/4127232686 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd584011af0 con 0x7fd58807f760 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.262+0000 7fd58f2e3700 1 -- 172.21.15.122:0/4127232686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 msgr2=0x7fd58807dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.262+0000 7fd58f2e3700 1 --2- 172.21.15.122:0/4127232686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58807dbc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd5840098a0 tx=0x7fd584009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.262+0000 7fd58f2e3700 1 -- 172.21.15.122:0/4127232686 shutdown_connections 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.262+0000 7fd58f2e3700 1 --2- 172.21.15.122:0/4127232686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58807dbc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.262+0000 7fd58f2e3700 1 -- 172.21.15.122:0/4127232686 >> 172.21.15.122:0/4127232686 conn(0x7fd5881043d0 msgr2=0x7fd5881067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.262+0000 7fd58f2e3700 1 -- 172.21.15.122:0/4127232686 shutdown_connections 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.262+0000 7fd58f2e3700 1 -- 172.21.15.122:0/4127232686 wait complete. 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.263+0000 7fd58f2e3700 1 Processor -- start 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.263+0000 7fd58f2e3700 1 -- start start 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.263+0000 7fd58f2e3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.263+0000 7fd58f2e3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd58810f3a0 con 0x7fd58807f760 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.263+0000 7fd58d07f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.263+0000 7fd58d07f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53560/0 (socket says 172.21.15.122:53560) 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.263+0000 7fd58d07f700 1 -- 172.21.15.122:0/311004112 learned_addr learned my addr 172.21.15.122:0/311004112 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.263+0000 7fd58d07f700 1 -- 172.21.15.122:0/311004112 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58819f4c0 con 0x7fd58807f760 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.264+0000 7fd58d07f700 1 --2- 172.21.15.122:0/311004112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58819ef80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd5840117b0 tx=0x7fd584011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.264+0000 7fd57e7fc700 1 -- 172.21.15.122:0/311004112 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd584003af0 con 0x7fd58807f760 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.264+0000 7fd57e7fc700 1 -- 172.21.15.122:0/311004112 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd584020070 con 0x7fd58807f760 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.264+0000 7fd58f2e3700 1 -- 172.21.15.122:0/311004112 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58819f750 con 0x7fd58807f760 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.264+0000 7fd57e7fc700 1 -- 172.21.15.122:0/311004112 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd584019630 con 0x7fd58807f760 2024-05-03T17:45:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.264+0000 7fd58f2e3700 1 -- 172.21.15.122:0/311004112 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5881a2440 con 0x7fd58807f760 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.265+0000 7fd57e7fc700 1 -- 172.21.15.122:0/311004112 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd58401a430 con 0x7fd58807f760 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.265+0000 7fd58f2e3700 1 -- 172.21.15.122:0/311004112 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58806a5c0 con 0x7fd58807f760 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.265+0000 7fd57e7fc700 1 --2- 172.21.15.122:0/311004112 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd57403e470 0x7fd574040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.265+0000 7fd57e7fc700 1 -- 172.21.15.122:0/311004112 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd584053aa0 con 0x7fd58807f760 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.265+0000 7fd58c87e700 1 --2- 172.21.15.122:0/311004112 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd57403e470 0x7fd574040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.265+0000 7fd58c87e700 1 --2- 172.21.15.122:0/311004112 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd57403e470 0x7fd574040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd578006fd0 tx=0x7fd578006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.270+0000 7fd57e7fc700 1 -- 172.21.15.122:0/311004112 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd584021e00 con 0x7fd58807f760 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:44:59.475+0000 7fd58f2e3700 1 -- 172.21.15.122:0/311004112 --> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi122", "addr": "172.21.15.122", "target": ["mon-mgr", ""]}) -- 0x7fd588105620 con 0x7fd57403e470 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:03.544+0000 7fd57e7fc700 1 -- 172.21.15.122:0/311004112 <== mgr.14120 v2:172.21.15.122:6800/1659741627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd588105620 con 0x7fd57403e470 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:03.547+0000 7fd58f2e3700 1 -- 172.21.15.122:0/311004112 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd57403e470 msgr2=0x7fd574040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:03.547+0000 7fd58f2e3700 1 --2- 172.21.15.122:0/311004112 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd57403e470 0x7fd574040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd578006fd0 tx=0x7fd578006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:03.547+0000 7fd58f2e3700 1 -- 172.21.15.122:0/311004112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 msgr2=0x7fd58819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:03.547+0000 7fd58f2e3700 1 --2- 172.21.15.122:0/311004112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58819ef80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd5840117b0 tx=0x7fd584011b70 comp rx=0 tx=0).stop 2024-05-03T17:45:04.132 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:03.547+0000 7fd58f2e3700 1 -- 172.21.15.122:0/311004112 shutdown_connections 2024-05-03T17:45:04.133 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:03.547+0000 7fd58f2e3700 1 --2- 172.21.15.122:0/311004112 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd57403e470 0x7fd574040930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:04.133 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:03.547+0000 7fd58f2e3700 1 --2- 172.21.15.122:0/311004112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58807f760 0x7fd58819ef80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:04.133 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:03.547+0000 7fd58f2e3700 1 -- 172.21.15.122:0/311004112 >> 172.21.15.122:0/311004112 conn(0x7fd5881043d0 msgr2=0x7fd588104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:04.133 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:03.547+0000 7fd58f2e3700 1 -- 172.21.15.122:0/311004112 shutdown_connections 2024-05-03T17:45:04.133 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:03.547+0000 7fd58f2e3700 1 -- 172.21.15.122:0/311004112 wait complete. 2024-05-03T17:45:04.133 INFO:teuthology.orchestra.run.smithi122.stdout:Deploying unmanaged mon service... 2024-05-03T17:45:04.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:04 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:45:04.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:04 smithi122 ceph-mon[29744]: Added host smithi122 2024-05-03T17:45:04.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:04 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:45:05.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.557+0000 7f82aa637700 1 Processor -- start 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.557+0000 7f82aa637700 1 -- start start 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.557+0000 7f82aa637700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a410bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.557+0000 7f82aa637700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f82a407d670 con 0x7f82a4109960 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.557+0000 7f82a3fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a410bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.557+0000 7f82a3fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a410bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53576/0 (socket says 172.21.15.122:53576) 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.557+0000 7f82a3fff700 1 -- 172.21.15.122:0/3586578401 learned_addr learned my addr 172.21.15.122:0/3586578401 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.558+0000 7f82a3fff700 1 -- 172.21.15.122:0/3586578401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a410c290 con 0x7f82a4109960 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.558+0000 7f82a3fff700 1 --2- 172.21.15.122:0/3586578401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a410bd50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82940098a0 tx=0x7f8294009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d90eb2d746f494 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.558+0000 7f82a2ffd700 1 -- 172.21.15.122:0/3586578401 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8294004160 con 0x7f82a4109960 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.558+0000 7f82a2ffd700 1 -- 172.21.15.122:0/3586578401 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8294004300 con 0x7f82a4109960 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.558+0000 7f82a2ffd700 1 -- 172.21.15.122:0/3586578401 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8294011af0 con 0x7f82a4109960 2024-05-03T17:45:05.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.558+0000 7f82aa637700 1 -- 172.21.15.122:0/3586578401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 msgr2=0x7f82a410bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:05.583 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.558+0000 7f82aa637700 1 --2- 172.21.15.122:0/3586578401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a410bd50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82940098a0 tx=0x7f8294009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:05.583 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.558+0000 7f82aa637700 1 -- 172.21.15.122:0/3586578401 shutdown_connections 2024-05-03T17:45:05.583 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.558+0000 7f82aa637700 1 --2- 172.21.15.122:0/3586578401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a410bd50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:05.583 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.558+0000 7f82aa637700 1 -- 172.21.15.122:0/3586578401 >> 172.21.15.122:0/3586578401 conn(0x7f82a41032f0 msgr2=0x7f82a4105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:05.583 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.559+0000 7f82aa637700 1 -- 172.21.15.122:0/3586578401 shutdown_connections 2024-05-03T17:45:05.583 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.559+0000 7f82aa637700 1 -- 172.21.15.122:0/3586578401 wait complete. 2024-05-03T17:45:05.583 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.559+0000 7f82aa637700 1 Processor -- start 2024-05-03T17:45:05.583 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.559+0000 7f82aa637700 1 -- start start 2024-05-03T17:45:05.583 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.559+0000 7f82aa637700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:05.583 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.559+0000 7f82aa637700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f82a410d190 con 0x7f82a4109960 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.560+0000 7f82a3fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.560+0000 7f82a3fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53592/0 (socket says 172.21.15.122:53592) 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.560+0000 7f82a3fff700 1 -- 172.21.15.122:0/1386193110 learned_addr learned my addr 172.21.15.122:0/1386193110 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.560+0000 7f82a3fff700 1 -- 172.21.15.122:0/1386193110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a419d2f0 con 0x7f82a4109960 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.560+0000 7f82a3fff700 1 --2- 172.21.15.122:0/1386193110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a419cdb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8294009fb0 tx=0x7f8294011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.560+0000 7f82a17fa700 1 -- 172.21.15.122:0/1386193110 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82940036a0 con 0x7f82a4109960 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.560+0000 7f82a17fa700 1 -- 172.21.15.122:0/1386193110 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f829401a430 con 0x7f82a4109960 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.560+0000 7f82aa637700 1 -- 172.21.15.122:0/1386193110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a419d580 con 0x7f82a4109960 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.560+0000 7f82a17fa700 1 -- 172.21.15.122:0/1386193110 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8294019550 con 0x7f82a4109960 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.560+0000 7f82aa637700 1 -- 172.21.15.122:0/1386193110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a41a0230 con 0x7f82a4109960 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.561+0000 7f82a17fa700 1 -- 172.21.15.122:0/1386193110 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f8294019850 con 0x7f82a4109960 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.561+0000 7f82a17fa700 1 --2- 172.21.15.122:0/1386193110 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f829003e420 0x7f82900408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.561+0000 7f82aa637700 1 -- 172.21.15.122:0/1386193110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a406a5c0 con 0x7f82a4109960 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.561+0000 7f82a17fa700 1 -- 172.21.15.122:0/1386193110 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f82940538d0 con 0x7f82a4109960 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.561+0000 7f82a37fe700 1 --2- 172.21.15.122:0/1386193110 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f829003e420 0x7f82900408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.562+0000 7f82a37fe700 1 --2- 172.21.15.122:0/1386193110 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f829003e420 0x7f82900408e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f828c006fd0 tx=0x7f828c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.566+0000 7f82a17fa700 1 -- 172.21.15.122:0/1386193110 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8294017e90 con 0x7f82a4109960 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.814+0000 7f82aa637700 1 -- 172.21.15.122:0/1386193110 --> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f82a4104470 con 0x7f829003e420 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.819+0000 7f82a17fa700 1 -- 172.21.15.122:0/1386193110 <== mgr.14120 v2:172.21.15.122:6800/1659741627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f82a4104470 con 0x7f829003e420 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.821+0000 7f82aa637700 1 -- 172.21.15.122:0/1386193110 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f829003e420 msgr2=0x7f82900408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.821+0000 7f82aa637700 1 --2- 172.21.15.122:0/1386193110 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f829003e420 0x7f82900408e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f828c006fd0 tx=0x7f828c006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.821+0000 7f82aa637700 1 -- 172.21.15.122:0/1386193110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 msgr2=0x7f82a419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.821+0000 7f82aa637700 1 --2- 172.21.15.122:0/1386193110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a419cdb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8294009fb0 tx=0x7f8294011940 comp rx=0 tx=0).stop 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.822+0000 7f82aa637700 1 -- 172.21.15.122:0/1386193110 shutdown_connections 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.822+0000 7f82aa637700 1 --2- 172.21.15.122:0/1386193110 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f829003e420 0x7f82900408e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.822+0000 7f82aa637700 1 --2- 172.21.15.122:0/1386193110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82a4109960 0x7f82a419cdb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.822+0000 7f82aa637700 1 -- 172.21.15.122:0/1386193110 >> 172.21.15.122:0/1386193110 conn(0x7f82a41032f0 msgr2=0x7f82a4103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.822+0000 7f82aa637700 1 -- 172.21.15.122:0/1386193110 shutdown_connections 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:04.822+0000 7f82aa637700 1 -- 172.21.15.122:0/1386193110 wait complete. 2024-05-03T17:45:05.585 INFO:teuthology.orchestra.run.smithi122.stdout:Deploying unmanaged mgr service... 2024-05-03T17:45:05.909 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:05 smithi122 ceph-mon[29744]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:45:05.909 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:05 smithi122 ceph-mon[29744]: Saving service mon spec with placement count:5 2024-05-03T17:45:05.909 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:05 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:45:07.536 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:07 smithi122 ceph-mon[29744]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:45:07.536 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:07 smithi122 ceph-mon[29744]: Saving service mgr spec with placement count:2 2024-05-03T17:45:07.536 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:07 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:45:07.536 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:07 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:45:07.544 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-03T17:45:07.544 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.183+0000 7f3f008fe700 1 Processor -- start 2024-05-03T17:45:07.544 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.183+0000 7f3f008fe700 1 -- start start 2024-05-03T17:45:07.544 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.184+0000 7f3f008fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc109230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:07.544 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.184+0000 7f3f008fe700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc109770 con 0x7f3efc108e10 2024-05-03T17:45:07.544 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.184+0000 7f3efa59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc109230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.184+0000 7f3efa59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc109230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53606/0 (socket says 172.21.15.122:53606) 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.184+0000 7f3efa59c700 1 -- 172.21.15.122:0/2153186024 learned_addr learned my addr 172.21.15.122:0/2153186024 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.184+0000 7f3efa59c700 1 -- 172.21.15.122:0/2153186024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc109f70 con 0x7f3efc108e10 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.184+0000 7f3efa59c700 1 --2- 172.21.15.122:0/2153186024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc109230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00098a0 tx=0x7f3ef0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c79bf184820f8a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.184+0000 7f3ef959a700 1 -- 172.21.15.122:0/2153186024 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0004160 con 0x7f3efc108e10 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.185+0000 7f3ef959a700 1 -- 172.21.15.122:0/2153186024 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3ef0004300 con 0x7f3efc108e10 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.185+0000 7f3ef959a700 1 -- 172.21.15.122:0/2153186024 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0011af0 con 0x7f3efc108e10 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.185+0000 7f3f008fe700 1 -- 172.21.15.122:0/2153186024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 msgr2=0x7f3efc109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.185+0000 7f3f008fe700 1 --2- 172.21.15.122:0/2153186024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc109230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00098a0 tx=0x7f3ef0009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.185+0000 7f3f008fe700 1 -- 172.21.15.122:0/2153186024 shutdown_connections 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.185+0000 7f3f008fe700 1 --2- 172.21.15.122:0/2153186024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc109230 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.185+0000 7f3f008fe700 1 -- 172.21.15.122:0/2153186024 >> 172.21.15.122:0/2153186024 conn(0x7f3efc1043d0 msgr2=0x7f3efc1067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.185+0000 7f3f008fe700 1 -- 172.21.15.122:0/2153186024 shutdown_connections 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.185+0000 7f3f008fe700 1 -- 172.21.15.122:0/2153186024 wait complete. 2024-05-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.186+0000 7f3f008fe700 1 Processor -- start 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.186+0000 7f3f008fe700 1 -- start start 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.186+0000 7f3f008fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.186+0000 7f3f008fe700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc10d3b0 con 0x7f3efc108e10 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.186+0000 7f3efa59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.186+0000 7f3efa59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53610/0 (socket says 172.21.15.122:53610) 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.186+0000 7f3efa59c700 1 -- 172.21.15.122:0/3933949604 learned_addr learned my addr 172.21.15.122:0/3933949604 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.186+0000 7f3efa59c700 1 -- 172.21.15.122:0/3933949604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc19d260 con 0x7f3efc108e10 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.187+0000 7f3efa59c700 1 --2- 172.21.15.122:0/3933949604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc19cd20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0011ac0 tx=0x7f3ef0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.187+0000 7f3eeb7fe700 1 -- 172.21.15.122:0/3933949604 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0003970 con 0x7f3efc108e10 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.187+0000 7f3eeb7fe700 1 -- 172.21.15.122:0/3933949604 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3ef0003b10 con 0x7f3efc108e10 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.187+0000 7f3eeb7fe700 1 -- 172.21.15.122:0/3933949604 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0024610 con 0x7f3efc108e10 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.187+0000 7f3f008fe700 1 -- 172.21.15.122:0/3933949604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc19d4f0 con 0x7f3efc108e10 2024-05-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.187+0000 7f3f008fe700 1 -- 172.21.15.122:0/3933949604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a01d0 con 0x7f3efc108e10 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.188+0000 7f3eeb7fe700 1 -- 172.21.15.122:0/3933949604 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3ef0024890 con 0x7f3efc108e10 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.188+0000 7f3f008fe700 1 -- 172.21.15.122:0/3933949604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc06a5c0 con 0x7f3efc108e10 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.188+0000 7f3eeb7fe700 1 --2- 172.21.15.122:0/3933949604 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f3ee403e470 0x7f3ee4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.188+0000 7f3eeb7fe700 1 -- 172.21.15.122:0/3933949604 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3ef0053b40 con 0x7f3efc108e10 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.188+0000 7f3ef9d9b700 1 --2- 172.21.15.122:0/3933949604 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f3ee403e470 0x7f3ee4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.188+0000 7f3ef9d9b700 1 --2- 172.21.15.122:0/3933949604 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f3ee403e470 0x7f3ee4040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3eec006fd0 tx=0x7f3eec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.193+0000 7f3eeb7fe700 1 -- 172.21.15.122:0/3933949604 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3ef002c350 con 0x7f3efc108e10 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.419+0000 7f3f008fe700 1 -- 172.21.15.122:0/3933949604 --> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3efc105580 con 0x7f3ee403e470 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.423+0000 7f3eeb7fe700 1 -- 172.21.15.122:0/3933949604 <== mgr.14120 v2:172.21.15.122:6800/1659741627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3efc105580 con 0x7f3ee403e470 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.426+0000 7f3f008fe700 1 -- 172.21.15.122:0/3933949604 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f3ee403e470 msgr2=0x7f3ee4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.426+0000 7f3f008fe700 1 --2- 172.21.15.122:0/3933949604 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f3ee403e470 0x7f3ee4040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3eec006fd0 tx=0x7f3eec006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.426+0000 7f3f008fe700 1 -- 172.21.15.122:0/3933949604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 msgr2=0x7f3efc19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.426+0000 7f3f008fe700 1 --2- 172.21.15.122:0/3933949604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc19cd20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0011ac0 tx=0x7f3ef0011ba0 comp rx=0 tx=0).stop 2024-05-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.426+0000 7f3f008fe700 1 -- 172.21.15.122:0/3933949604 shutdown_connections 2024-05-03T17:45:07.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.426+0000 7f3f008fe700 1 --2- 172.21.15.122:0/3933949604 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f3ee403e470 0x7f3ee4040930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:07.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.426+0000 7f3f008fe700 1 --2- 172.21.15.122:0/3933949604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3efc108e10 0x7f3efc19cd20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:07.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.426+0000 7f3f008fe700 1 -- 172.21.15.122:0/3933949604 >> 172.21.15.122:0/3933949604 conn(0x7f3efc1043d0 msgr2=0x7f3efc104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:07.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.426+0000 7f3f008fe700 1 -- 172.21.15.122:0/3933949604 shutdown_connections 2024-05-03T17:45:07.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:06.426+0000 7f3f008fe700 1 -- 172.21.15.122:0/3933949604 wait complete. 2024-05-03T17:45:08.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.952+0000 7f33a66a3700 1 Processor -- start 2024-05-03T17:45:08.760 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.952+0000 7f33a66a3700 1 -- start start 2024-05-03T17:45:08.760 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.953+0000 7f33a66a3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a0109230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:08.760 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.953+0000 7f33a66a3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0109770 con 0x7f33a0108e10 2024-05-03T17:45:08.760 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.953+0000 7f339ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a0109230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:08.760 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.953+0000 7f339ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a0109230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53618/0 (socket says 172.21.15.122:53618) 2024-05-03T17:45:08.760 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.953+0000 7f339ffff700 1 -- 172.21.15.122:0/97082880 learned_addr learned my addr 172.21.15.122:0/97082880 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:08.760 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.953+0000 7f339ffff700 1 -- 172.21.15.122:0/97082880 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a0109f70 con 0x7f33a0108e10 2024-05-03T17:45:08.760 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.953+0000 7f339ffff700 1 --2- 172.21.15.122:0/97082880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a0109230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f33900098a0 tx=0x7f3390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff086730853fbee7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:08.760 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.953+0000 7f339effd700 1 -- 172.21.15.122:0/97082880 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3390004160 con 0x7f33a0108e10 2024-05-03T17:45:08.760 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.953+0000 7f339effd700 1 -- 172.21.15.122:0/97082880 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3390004300 con 0x7f33a0108e10 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.953+0000 7f339effd700 1 -- 172.21.15.122:0/97082880 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3390011a20 con 0x7f33a0108e10 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.954+0000 7f33a66a3700 1 -- 172.21.15.122:0/97082880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 msgr2=0x7f33a0109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.954+0000 7f33a66a3700 1 --2- 172.21.15.122:0/97082880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a0109230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f33900098a0 tx=0x7f3390009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.954+0000 7f33a66a3700 1 -- 172.21.15.122:0/97082880 shutdown_connections 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.954+0000 7f33a66a3700 1 --2- 172.21.15.122:0/97082880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a0109230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.954+0000 7f33a66a3700 1 -- 172.21.15.122:0/97082880 >> 172.21.15.122:0/97082880 conn(0x7f33a01043d0 msgr2=0x7f33a01067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.954+0000 7f33a66a3700 1 -- 172.21.15.122:0/97082880 shutdown_connections 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.954+0000 7f33a66a3700 1 -- 172.21.15.122:0/97082880 wait complete. 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.955+0000 7f33a66a3700 1 Processor -- start 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.955+0000 7f33a66a3700 1 -- start start 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.955+0000 7f33a66a3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.955+0000 7f33a66a3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33a010d3b0 con 0x7f33a0108e10 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.955+0000 7f339ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.955+0000 7f339ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53626/0 (socket says 172.21.15.122:53626) 2024-05-03T17:45:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.955+0000 7f339ffff700 1 -- 172.21.15.122:0/4293147432 learned_addr learned my addr 172.21.15.122:0/4293147432 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.955+0000 7f339ffff700 1 -- 172.21.15.122:0/4293147432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a019d220 con 0x7f33a0108e10 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.956+0000 7f339ffff700 1 --2- 172.21.15.122:0/4293147432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a019cce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33900116e0 tx=0x7f3390011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.956+0000 7f339d7fa700 1 -- 172.21.15.122:0/4293147432 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3390003b20 con 0x7f33a0108e10 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.956+0000 7f339d7fa700 1 -- 172.21.15.122:0/4293147432 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339001b8f0 con 0x7f33a0108e10 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.956+0000 7f33a66a3700 1 -- 172.21.15.122:0/4293147432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a019d4b0 con 0x7f33a0108e10 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.956+0000 7f339d7fa700 1 -- 172.21.15.122:0/4293147432 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33900238c0 con 0x7f33a0108e10 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.956+0000 7f33a66a3700 1 -- 172.21.15.122:0/4293147432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a01a0190 con 0x7f33a0108e10 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.957+0000 7f339d7fa700 1 -- 172.21.15.122:0/4293147432 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3390023b70 con 0x7f33a0108e10 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.957+0000 7f339d7fa700 1 --2- 172.21.15.122:0/4293147432 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f338803e420 0x7f33880408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.957+0000 7f339d7fa700 1 -- 172.21.15.122:0/4293147432 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3390053f70 con 0x7f33a0108e10 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.957+0000 7f339f7fe700 1 --2- 172.21.15.122:0/4293147432 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f338803e420 0x7f33880408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.957+0000 7f33a66a3700 1 -- 172.21.15.122:0/4293147432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a010d530 con 0x7f33a0108e10 2024-05-03T17:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.957+0000 7f339f7fe700 1 --2- 172.21.15.122:0/4293147432 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f338803e420 0x7f33880408e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3394006fd0 tx=0x7f3394006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:07.962+0000 7f339d7fa700 1 -- 172.21.15.122:0/4293147432 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f339001f030 con 0x7f33a0108e10 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.174+0000 7f33a66a3700 1 -- 172.21.15.122:0/4293147432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f33a006a5c0 con 0x7f33a0108e10 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.179+0000 7f339d7fa700 1 -- 172.21.15.122:0/4293147432 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f339000b3e0 con 0x7f33a0108e10 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.181+0000 7f33a66a3700 1 -- 172.21.15.122:0/4293147432 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f338803e420 msgr2=0x7f33880408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.181+0000 7f33a66a3700 1 --2- 172.21.15.122:0/4293147432 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f338803e420 0x7f33880408e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3394006fd0 tx=0x7f3394006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.181+0000 7f33a66a3700 1 -- 172.21.15.122:0/4293147432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 msgr2=0x7f33a019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.181+0000 7f33a66a3700 1 --2- 172.21.15.122:0/4293147432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a019cce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33900116e0 tx=0x7f3390011aa0 comp rx=0 tx=0).stop 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.182+0000 7f33a66a3700 1 -- 172.21.15.122:0/4293147432 shutdown_connections 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.182+0000 7f33a66a3700 1 --2- 172.21.15.122:0/4293147432 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f338803e420 0x7f33880408e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.182+0000 7f33a66a3700 1 --2- 172.21.15.122:0/4293147432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33a0108e10 0x7f33a019cce0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.182+0000 7f33a66a3700 1 -- 172.21.15.122:0/4293147432 >> 172.21.15.122:0/4293147432 conn(0x7f33a01043d0 msgr2=0x7f33a0104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.182+0000 7f33a66a3700 1 -- 172.21.15.122:0/4293147432 shutdown_connections 2024-05-03T17:45:08.763 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:08.182+0000 7f33a66a3700 1 -- 172.21.15.122:0/4293147432 wait complete. 2024-05-03T17:45:09.356 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:09 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:45:09.357 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:09 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/4293147432' entity='client.admin' 2024-05-03T17:45:09.882 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.136+0000 7fd0325a4700 1 Processor -- start 2024-05-03T17:45:09.882 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.136+0000 7fd0325a4700 1 -- start start 2024-05-03T17:45:09.882 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.136+0000 7fd0325a4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c108170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:09.882 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.136+0000 7fd0325a4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c108740 con 0x7fd02c107d50 2024-05-03T17:45:09.882 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.136+0000 7fd02bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:09.882 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.136+0000 7fd02bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c108170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44648/0 (socket says 172.21.15.122:44648) 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.136+0000 7fd02bfff700 1 -- 172.21.15.122:0/4054542617 learned_addr learned my addr 172.21.15.122:0/4054542617 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.137+0000 7fd02bfff700 1 -- 172.21.15.122:0/4054542617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02c108f50 con 0x7fd02c107d50 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.137+0000 7fd02bfff700 1 --2- 172.21.15.122:0/4054542617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c108170 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0098a0 tx=0x7fd01c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24d1b72df583a44 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.137+0000 7fd02affd700 1 -- 172.21.15.122:0/4054542617 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd01c004160 con 0x7fd02c107d50 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.137+0000 7fd02affd700 1 -- 172.21.15.122:0/4054542617 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01c004300 con 0x7fd02c107d50 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.137+0000 7fd02affd700 1 -- 172.21.15.122:0/4054542617 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd01c011a20 con 0x7fd02c107d50 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.138+0000 7fd0325a4700 1 -- 172.21.15.122:0/4054542617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 msgr2=0x7fd02c108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.138+0000 7fd0325a4700 1 --2- 172.21.15.122:0/4054542617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c108170 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0098a0 tx=0x7fd01c009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.138+0000 7fd0325a4700 1 -- 172.21.15.122:0/4054542617 shutdown_connections 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.138+0000 7fd0325a4700 1 --2- 172.21.15.122:0/4054542617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c108170 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.138+0000 7fd0325a4700 1 -- 172.21.15.122:0/4054542617 >> 172.21.15.122:0/4054542617 conn(0x7fd02c1032f0 msgr2=0x7fd02c105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:09.883 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.138+0000 7fd0325a4700 1 -- 172.21.15.122:0/4054542617 shutdown_connections 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.138+0000 7fd0325a4700 1 -- 172.21.15.122:0/4054542617 wait complete. 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.139+0000 7fd0325a4700 1 Processor -- start 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.139+0000 7fd0325a4700 1 -- start start 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.139+0000 7fd0325a4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.139+0000 7fd0325a4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c10c300 con 0x7fd02c107d50 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.139+0000 7fd02bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.139+0000 7fd02bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c19bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44650/0 (socket says 172.21.15.122:44650) 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.139+0000 7fd02bfff700 1 -- 172.21.15.122:0/2968022231 learned_addr learned my addr 172.21.15.122:0/2968022231 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.140+0000 7fd02bfff700 1 -- 172.21.15.122:0/2968022231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02c19c200 con 0x7fd02c107d50 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.140+0000 7fd02bfff700 1 --2- 172.21.15.122:0/2968022231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c19bcc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd01c000c00 tx=0x7fd01c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.140+0000 7fd0297fa700 1 -- 172.21.15.122:0/2968022231 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd01c003aa0 con 0x7fd02c107d50 2024-05-03T17:45:09.884 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.140+0000 7fd0297fa700 1 -- 172.21.15.122:0/2968022231 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01c01aa10 con 0x7fd02c107d50 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.140+0000 7fd0325a4700 1 -- 172.21.15.122:0/2968022231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02c19c490 con 0x7fd02c107d50 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.140+0000 7fd0297fa700 1 -- 172.21.15.122:0/2968022231 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd01c023a10 con 0x7fd02c107d50 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.140+0000 7fd0325a4700 1 -- 172.21.15.122:0/2968022231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02c07bfe0 con 0x7fd02c107d50 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.141+0000 7fd0297fa700 1 -- 172.21.15.122:0/2968022231 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd01c023c50 con 0x7fd02c107d50 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.141+0000 7fd0325a4700 1 -- 172.21.15.122:0/2968022231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd02c06a5c0 con 0x7fd02c107d50 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.142+0000 7fd0297fa700 1 --2- 172.21.15.122:0/2968022231 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd01403e470 0x7fd014040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.142+0000 7fd0297fa700 1 -- 172.21.15.122:0/2968022231 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd01c055610 con 0x7fd02c107d50 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.142+0000 7fd02b7fe700 1 --2- 172.21.15.122:0/2968022231 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd01403e470 0x7fd014040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.142+0000 7fd02b7fe700 1 --2- 172.21.15.122:0/2968022231 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd01403e470 0x7fd014040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd020006fd0 tx=0x7fd020006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.150+0000 7fd0297fa700 1 -- 172.21.15.122:0/2968022231 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd01c055370 con 0x7fd02c107d50 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.360+0000 7fd0325a4700 1 -- 172.21.15.122:0/2968022231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fd02c07c640 con 0x7fd02c107d50 2024-05-03T17:45:09.885 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.364+0000 7fd0297fa700 1 -- 172.21.15.122:0/2968022231 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fd01c025db0 con 0x7fd02c107d50 2024-05-03T17:45:09.886 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.366+0000 7fd0325a4700 1 -- 172.21.15.122:0/2968022231 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd01403e470 msgr2=0x7fd014040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:09.886 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.366+0000 7fd0325a4700 1 --2- 172.21.15.122:0/2968022231 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd01403e470 0x7fd014040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd020006fd0 tx=0x7fd020006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:09.886 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.366+0000 7fd0325a4700 1 -- 172.21.15.122:0/2968022231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 msgr2=0x7fd02c19bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:09.886 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.366+0000 7fd0325a4700 1 --2- 172.21.15.122:0/2968022231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c19bcc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd01c000c00 tx=0x7fd01c011aa0 comp rx=0 tx=0).stop 2024-05-03T17:45:09.886 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.366+0000 7fd0325a4700 1 -- 172.21.15.122:0/2968022231 shutdown_connections 2024-05-03T17:45:09.886 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.366+0000 7fd0325a4700 1 --2- 172.21.15.122:0/2968022231 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd01403e470 0x7fd014040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:09.886 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.366+0000 7fd0325a4700 1 --2- 172.21.15.122:0/2968022231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd02c107d50 0x7fd02c19bcc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:09.886 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.366+0000 7fd0325a4700 1 -- 172.21.15.122:0/2968022231 >> 172.21.15.122:0/2968022231 conn(0x7fd02c1032f0 msgr2=0x7fd02c103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:09.886 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.366+0000 7fd0325a4700 1 -- 172.21.15.122:0/2968022231 shutdown_connections 2024-05-03T17:45:09.887 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:09.366+0000 7fd0325a4700 1 -- 172.21.15.122:0/2968022231 wait complete. 2024-05-03T17:45:09.887 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling the dashboard module... 2024-05-03T17:45:10.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:10 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2968022231' entity='client.admin' 2024-05-03T17:45:11.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:11 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1298606138' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-03T17:45:11.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:11 smithi122 ceph-mon[29744]: from='mgr.14120 172.21.15.122:0/2395578426' entity='mgr.x' 2024-05-03T17:45:11.461 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:11 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: ignoring --setuser ceph since I am not root 2024-05-03T17:45:11.461 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:11 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: ignoring --setgroup ceph since I am not root 2024-05-03T17:45:11.830 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:11 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:11.547+0000 7f37e1b12200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T17:45:11.830 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:11 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:11.733+0000 7f37e1b12200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T17:45:11.830 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:11 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:11.834+0000 7f37e1b12200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T17:45:11.999 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.316+0000 7f45cd40a700 1 Processor -- start 2024-05-03T17:45:11.999 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.316+0000 7f45cd40a700 1 -- start start 2024-05-03T17:45:11.999 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.316+0000 7f45cd40a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c8108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:11.999 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.316+0000 7f45cd40a700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81086f0 con 0x7f45c8107d00 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.317+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c8108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.317+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c8108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44666/0 (socket says 172.21.15.122:44666) 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.317+0000 7f45c6ffd700 1 -- 172.21.15.122:0/494517000 learned_addr learned my addr 172.21.15.122:0/494517000 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.317+0000 7f45c6ffd700 1 -- 172.21.15.122:0/494517000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c8108f00 con 0x7f45c8107d00 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.317+0000 7f45c6ffd700 1 --2- 172.21.15.122:0/494517000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c8108120 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f45b80098a0 tx=0x7f45b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf39436a3448071 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.317+0000 7f45c67fc700 1 -- 172.21.15.122:0/494517000 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45b8004160 con 0x7f45c8107d00 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.317+0000 7f45c67fc700 1 -- 172.21.15.122:0/494517000 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b8004300 con 0x7f45c8107d00 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.317+0000 7f45c67fc700 1 -- 172.21.15.122:0/494517000 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45b8011a20 con 0x7f45c8107d00 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.318+0000 7f45cd40a700 1 -- 172.21.15.122:0/494517000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 msgr2=0x7f45c8108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.318+0000 7f45cd40a700 1 --2- 172.21.15.122:0/494517000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c8108120 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f45b80098a0 tx=0x7f45b8009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.318+0000 7f45cd40a700 1 -- 172.21.15.122:0/494517000 shutdown_connections 2024-05-03T17:45:12.000 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.318+0000 7f45cd40a700 1 --2- 172.21.15.122:0/494517000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c8108120 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.318+0000 7f45cd40a700 1 -- 172.21.15.122:0/494517000 >> 172.21.15.122:0/494517000 conn(0x7f45c81032c0 msgr2=0x7f45c81056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.318+0000 7f45cd40a700 1 -- 172.21.15.122:0/494517000 shutdown_connections 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.318+0000 7f45cd40a700 1 -- 172.21.15.122:0/494517000 wait complete. 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.318+0000 7f45cd40a700 1 Processor -- start 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45cd40a700 1 -- start start 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45cd40a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c807d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45cd40a700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45c810c2b0 con 0x7f45c8107d00 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c807d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c807d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44678/0 (socket says 172.21.15.122:44678) 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45c6ffd700 1 -- 172.21.15.122:0/1298606138 learned_addr learned my addr 172.21.15.122:0/1298606138 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45c6ffd700 1 -- 172.21.15.122:0/1298606138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c819fac0 con 0x7f45c8107d00 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45c6ffd700 1 --2- 172.21.15.122:0/1298606138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c807d2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f45b80119f0 tx=0x7f45b8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45c4ff9700 1 -- 172.21.15.122:0/1298606138 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45b80039e0 con 0x7f45c8107d00 2024-05-03T17:45:12.001 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45cd40a700 1 -- 172.21.15.122:0/1298606138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c819fc90 con 0x7f45c8107d00 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45c4ff9700 1 -- 172.21.15.122:0/1298606138 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b801b8f0 con 0x7f45c8107d00 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45c4ff9700 1 -- 172.21.15.122:0/1298606138 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45b80237a0 con 0x7f45c8107d00 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.319+0000 7f45cd40a700 1 -- 172.21.15.122:0/1298606138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c81a0030 con 0x7f45c8107d00 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.320+0000 7f45c4ff9700 1 -- 172.21.15.122:0/1298606138 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f45b802c3f0 con 0x7f45c8107d00 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.320+0000 7f45c4ff9700 1 --2- 172.21.15.122:0/1298606138 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f45b403e420 0x7f45b40408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.320+0000 7f45cd40a700 1 -- 172.21.15.122:0/1298606138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c806a5c0 con 0x7f45c8107d00 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.320+0000 7f45c4ff9700 1 -- 172.21.15.122:0/1298606138 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f45b8053e10 con 0x7f45c8107d00 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.320+0000 7f45be7fc700 1 --2- 172.21.15.122:0/1298606138 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f45b403e420 0x7f45b40408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.321+0000 7f45be7fc700 1 --2- 172.21.15.122:0/1298606138 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f45b403e420 0x7f45b40408e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f45b0006fd0 tx=0x7f45b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.325+0000 7f45c4ff9700 1 -- 172.21.15.122:0/1298606138 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f45b801f030 con 0x7f45c8107d00 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:10.592+0000 7f45cd40a700 1 -- 172.21.15.122:0/1298606138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f45c81a0490 con 0x7f45c8107d00 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.367+0000 7f45c4ff9700 1 -- 172.21.15.122:0/1298606138 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f45b80202b0 con 0x7f45c8107d00 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.369+0000 7f45c4ff9700 1 -- 172.21.15.122:0/1298606138 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f45b804fcf0 con 0x7f45c8107d00 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.370+0000 7f45cd40a700 1 -- 172.21.15.122:0/1298606138 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f45b403e420 msgr2=0x7f45b40408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:12.002 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.370+0000 7f45cd40a700 1 --2- 172.21.15.122:0/1298606138 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f45b403e420 0x7f45b40408e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f45b0006fd0 tx=0x7f45b0006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:12.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.370+0000 7f45cd40a700 1 -- 172.21.15.122:0/1298606138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 msgr2=0x7f45c807d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:12.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.370+0000 7f45cd40a700 1 --2- 172.21.15.122:0/1298606138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c807d2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f45b80119f0 tx=0x7f45b8011ad0 comp rx=0 tx=0).stop 2024-05-03T17:45:12.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.370+0000 7f45cd40a700 1 -- 172.21.15.122:0/1298606138 shutdown_connections 2024-05-03T17:45:12.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.370+0000 7f45cd40a700 1 --2- 172.21.15.122:0/1298606138 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f45b403e420 0x7f45b40408e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:12.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.370+0000 7f45cd40a700 1 --2- 172.21.15.122:0/1298606138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45c8107d00 0x7f45c807d2f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:12.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.370+0000 7f45cd40a700 1 -- 172.21.15.122:0/1298606138 >> 172.21.15.122:0/1298606138 conn(0x7f45c81032c0 msgr2=0x7f45c8103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:12.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.370+0000 7f45cd40a700 1 -- 172.21.15.122:0/1298606138 shutdown_connections 2024-05-03T17:45:12.003 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:11.370+0000 7f45cd40a700 1 -- 172.21.15.122:0/1298606138 wait complete. 2024-05-03T17:45:12.394 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:12 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1298606138' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-03T17:45:12.395 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:12 smithi122 ceph-mon[29744]: mgrmap e9: x(active, since 19s) 2024-05-03T17:45:12.395 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:12 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:12.147+0000 7f37e1b12200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T17:45:12.395 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:12 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:12.234+0000 7f37e1b12200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T17:45:13.306 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-05-03T17:45:13.306 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.444+0000 7fd7f4239700 1 Processor -- start 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.444+0000 7fd7f4239700 1 -- start start 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.444+0000 7fd7f4239700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec10bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.444+0000 7fd7f4239700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec07d6a0 con 0x7fd7ec109900 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.445+0000 7fd7f1fd5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec10bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.445+0000 7fd7f1fd5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec10bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44688/0 (socket says 172.21.15.122:44688) 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.445+0000 7fd7f1fd5700 1 -- 172.21.15.122:0/2670261124 learned_addr learned my addr 172.21.15.122:0/2670261124 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:13.307 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.445+0000 7fd7f1fd5700 1 -- 172.21.15.122:0/2670261124 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ec10c230 con 0x7fd7ec109900 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.445+0000 7fd7f1fd5700 1 --2- 172.21.15.122:0/2670261124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec10bcf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0098a0 tx=0x7fd7dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2dedf27a0aa2ad server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.445+0000 7fd7f0fd3700 1 -- 172.21.15.122:0/2670261124 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7dc004160 con 0x7fd7ec109900 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.445+0000 7fd7f0fd3700 1 -- 172.21.15.122:0/2670261124 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7dc004300 con 0x7fd7ec109900 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.445+0000 7fd7f0fd3700 1 -- 172.21.15.122:0/2670261124 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7dc011a20 con 0x7fd7ec109900 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.446+0000 7fd7f4239700 1 -- 172.21.15.122:0/2670261124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 msgr2=0x7fd7ec10bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.446+0000 7fd7f4239700 1 --2- 172.21.15.122:0/2670261124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec10bcf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0098a0 tx=0x7fd7dc009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.446+0000 7fd7f4239700 1 -- 172.21.15.122:0/2670261124 shutdown_connections 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.446+0000 7fd7f4239700 1 --2- 172.21.15.122:0/2670261124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec10bcf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.446+0000 7fd7f4239700 1 -- 172.21.15.122:0/2670261124 >> 172.21.15.122:0/2670261124 conn(0x7fd7ec103310 msgr2=0x7fd7ec105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.446+0000 7fd7f4239700 1 -- 172.21.15.122:0/2670261124 shutdown_connections 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.446+0000 7fd7f4239700 1 -- 172.21.15.122:0/2670261124 wait complete. 2024-05-03T17:45:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7f4239700 1 Processor -- start 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7f4239700 1 -- start start 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7f4239700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7f4239700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec10d1d0 con 0x7fd7ec109900 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7f1fd5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7f1fd5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44690/0 (socket says 172.21.15.122:44690) 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7f1fd5700 1 -- 172.21.15.122:0/3459191876 learned_addr learned my addr 172.21.15.122:0/3459191876 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7f1fd5700 1 -- 172.21.15.122:0/3459191876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ec19f480 con 0x7fd7ec109900 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7f1fd5700 1 --2- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec19ef40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc004660 tx=0x7fd7dc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7e2ffd700 1 -- 172.21.15.122:0/3459191876 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7dc003aa0 con 0x7fd7ec109900 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7e2ffd700 1 -- 172.21.15.122:0/3459191876 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7dc01b8f0 con 0x7fd7ec109900 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7f4239700 1 -- 172.21.15.122:0/3459191876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ec19f710 con 0x7fd7ec109900 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.447+0000 7fd7e2ffd700 1 -- 172.21.15.122:0/3459191876 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7dc023900 con 0x7fd7ec109900 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.448+0000 7fd7f4239700 1 -- 172.21.15.122:0/3459191876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ec1a2400 con 0x7fd7ec109900 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.448+0000 7fd7e2ffd700 1 -- 172.21.15.122:0/3459191876 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fd7dc023b70 con 0x7fd7ec109900 2024-05-03T17:45:13.309 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.449+0000 7fd7e2ffd700 1 --2- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd7d803e520 0x7fd7d80409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.449+0000 7fd7f4239700 1 -- 172.21.15.122:0/3459191876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ec06a5c0 con 0x7fd7ec109900 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.449+0000 7fd7f17d4700 1 -- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd7d803e520 msgr2=0x7fd7d80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1659741627 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.449+0000 7fd7e2ffd700 1 -- 172.21.15.122:0/3459191876 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd7dc053db0 con 0x7fd7ec109900 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.449+0000 7fd7f17d4700 1 --2- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd7d803e520 0x7fd7d80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.454+0000 7fd7e2ffd700 1 -- 172.21.15.122:0/3459191876 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd7dc01f030 con 0x7fd7ec109900 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.650+0000 7fd7f17d4700 1 -- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd7d803e520 msgr2=0x7fd7d80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1659741627 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.650+0000 7fd7f17d4700 1 --2- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd7d803e520 0x7fd7d80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.760+0000 7fd7f4239700 1 -- 172.21.15.122:0/3459191876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd7ec1a29c0 con 0x7fd7ec109900 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.761+0000 7fd7e2ffd700 1 -- 172.21.15.122:0/3459191876 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fd7dc0292a0 con 0x7fd7ec109900 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.763+0000 7fd7f4239700 1 -- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd7d803e520 msgr2=0x7fd7d80409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.763+0000 7fd7f4239700 1 --2- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd7d803e520 0x7fd7d80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:13.310 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.763+0000 7fd7f4239700 1 -- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 msgr2=0x7fd7ec19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:13.311 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.763+0000 7fd7f4239700 1 --2- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec19ef40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc004660 tx=0x7fd7dc011aa0 comp rx=0 tx=0).stop 2024-05-03T17:45:13.311 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.763+0000 7fd7f4239700 1 -- 172.21.15.122:0/3459191876 shutdown_connections 2024-05-03T17:45:13.311 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.763+0000 7fd7f4239700 1 --2- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7fd7d803e520 0x7fd7d80409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:13.311 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.763+0000 7fd7f4239700 1 --2- 172.21.15.122:0/3459191876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7ec109900 0x7fd7ec19ef40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:13.311 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.763+0000 7fd7f4239700 1 -- 172.21.15.122:0/3459191876 >> 172.21.15.122:0/3459191876 conn(0x7fd7ec103310 msgr2=0x7fd7ec103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:13.311 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.763+0000 7fd7f4239700 1 -- 172.21.15.122:0/3459191876 shutdown_connections 2024-05-03T17:45:13.311 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:12.763+0000 7fd7f4239700 1 -- 172.21.15.122:0/3459191876 wait complete. 2024-05-03T17:45:13.311 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for the mgr to restart... 2024-05-03T17:45:13.311 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr epoch 9... 2024-05-03T17:45:13.428 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:13 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3459191876' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-03T17:45:13.710 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:13 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:13.463+0000 7f37e1b12200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T17:45:13.711 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:13 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:13.637+0000 7f37e1b12200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T17:45:14.100 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:13 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:13.814+0000 7f37e1b12200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T17:45:14.362 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:14 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:14.104+0000 7f37e1b12200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T17:45:14.362 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:14 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:14.198+0000 7f37e1b12200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T17:45:14.710 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:14 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:14.398+0000 7f37e1b12200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T17:45:14.711 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:14 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:14.597+0000 7f37e1b12200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T17:45:15.460 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:15 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:15.189+0000 7f37e1b12200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T17:45:15.461 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:15 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:15.410+0000 7f37e1b12200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T17:45:16.210 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:15 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:15.836+0000 7f37e1b12200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T17:45:16.211 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:15 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:15.934+0000 7f37e1b12200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T17:45:16.655 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:16.305+0000 7f37e1b12200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T17:45:16.960 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:16.659+0000 7f37e1b12200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T17:45:16.960 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:16.751+0000 7f37e1b12200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T17:45:17.428 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:16.995+0000 7f37e1b12200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T17:45:17.428 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:17.129+0000 7f37e1b12200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T17:45:17.428 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:17.432+0000 7f37e1b12200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T17:45:17.710 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:17.548+0000 7f37e1b12200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T17:45:17.710 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:17.640+0000 7f37e1b12200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T17:45:18.501 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:18 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:18.234+0000 7f37e1b12200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T17:45:18.501 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:45:18 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:45:18.452+0000 7f37e1b12200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: Active manager daemon x restarted 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: Activating manager daemon x 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: osdmap e3: 0 total, 0 up, 0 in 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: mgrmap e10: x(active, starting, since 0.00424144s) 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: Manager daemon x is now available 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:45:18.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T17:45:19.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T17:45:19.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:19 smithi122 ceph-mon[29744]: [03/May/2024:17:45:18] ENGINE Bus STARTING 2024-05-03T17:45:19.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:19 smithi122 ceph-mon[29744]: [03/May/2024:17:45:19] ENGINE Serving on https://172.21.15.122:7150 2024-05-03T17:45:19.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:19 smithi122 ceph-mon[29744]: [03/May/2024:17:45:19] ENGINE Serving on http://172.21.15.122:8765 2024-05-03T17:45:19.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:19 smithi122 ceph-mon[29744]: [03/May/2024:17:45:19] ENGINE Bus STARTED 2024-05-03T17:45:19.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:19 smithi122 ceph-mon[29744]: mgrmap e11: x(active, since 1.00599s) 2024-05-03T17:45:20.401 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-05-03T17:45:20.401 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.759+0000 7f0253a13700 1 Processor -- start 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.759+0000 7f0253a13700 1 -- start start 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.759+0000 7f0253a13700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c07dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.759+0000 7f0253a13700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f024c07e190 con 0x7f024c07f710 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.760+0000 7f02517af700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c07dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.760+0000 7f02517af700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c07dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44696/0 (socket says 172.21.15.122:44696) 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.760+0000 7f02517af700 1 -- 172.21.15.122:0/4021382795 learned_addr learned my addr 172.21.15.122:0/4021382795 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.760+0000 7f02517af700 1 -- 172.21.15.122:0/4021382795 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024c07e310 con 0x7f024c07f710 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.760+0000 7f02517af700 1 --2- 172.21.15.122:0/4021382795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c07dbc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f023c0098a0 tx=0x7f023c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dffdaa198d77fd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.760+0000 7f0243fff700 1 -- 172.21.15.122:0/4021382795 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f023c004160 con 0x7f024c07f710 2024-05-03T17:45:20.402 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.760+0000 7f0243fff700 1 -- 172.21.15.122:0/4021382795 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f023c004300 con 0x7f024c07f710 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.760+0000 7f0243fff700 1 -- 172.21.15.122:0/4021382795 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f023c011a20 con 0x7f024c07f710 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.761+0000 7f0253a13700 1 -- 172.21.15.122:0/4021382795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 msgr2=0x7f024c07dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.761+0000 7f0253a13700 1 --2- 172.21.15.122:0/4021382795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c07dbc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f023c0098a0 tx=0x7f023c009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.761+0000 7f0253a13700 1 -- 172.21.15.122:0/4021382795 shutdown_connections 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.761+0000 7f0253a13700 1 --2- 172.21.15.122:0/4021382795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c07dbc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.761+0000 7f0253a13700 1 -- 172.21.15.122:0/4021382795 >> 172.21.15.122:0/4021382795 conn(0x7f024c104410 msgr2=0x7f024c106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.761+0000 7f0253a13700 1 -- 172.21.15.122:0/4021382795 shutdown_connections 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.761+0000 7f0253a13700 1 -- 172.21.15.122:0/4021382795 wait complete. 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.762+0000 7f0253a13700 1 Processor -- start 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.762+0000 7f0253a13700 1 -- start start 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.762+0000 7f0253a13700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.762+0000 7f0253a13700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f024c10f3b0 con 0x7f024c07f710 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.762+0000 7f02517af700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.762+0000 7f02517af700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44712/0 (socket says 172.21.15.122:44712) 2024-05-03T17:45:20.403 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.762+0000 7f02517af700 1 -- 172.21.15.122:0/946386765 learned_addr learned my addr 172.21.15.122:0/946386765 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.762+0000 7f02517af700 1 -- 172.21.15.122:0/946386765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024c19f480 con 0x7f024c07f710 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.762+0000 7f02517af700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c19ef40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f023c0116e0 tx=0x7f023c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.762+0000 7f02427fc700 1 -- 172.21.15.122:0/946386765 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f023c003b20 con 0x7f024c07f710 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.763+0000 7f0253a13700 1 -- 172.21.15.122:0/946386765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024c19f710 con 0x7f024c07f710 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.763+0000 7f02427fc700 1 -- 172.21.15.122:0/946386765 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f023c01b8f0 con 0x7f024c07f710 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.763+0000 7f02427fc700 1 -- 172.21.15.122:0/946386765 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f023c0238c0 con 0x7f024c07f710 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.763+0000 7f0253a13700 1 -- 172.21.15.122:0/946386765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f024c1a23b0 con 0x7f024c07f710 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.764+0000 7f02427fc700 1 -- 172.21.15.122:0/946386765 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f023c023b90 con 0x7f024c07f710 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.764+0000 7f02427fc700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 0x7f0238040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.764+0000 7f0253a13700 1 -- 172.21.15.122:0/946386765 --> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f024c06a5c0 con 0x7f023803e470 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.764+0000 7f0250fae700 1 -- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 msgr2=0x7f0238040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1659741627 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.764+0000 7f02427fc700 1 -- 172.21.15.122:0/946386765 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f023c054090 con 0x7f024c07f710 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.764+0000 7f0250fae700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 0x7f0238040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T17:45:20.404 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.965+0000 7f0250fae700 1 -- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 msgr2=0x7f0238040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1659741627 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:13.965+0000 7f0250fae700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 0x7f0238040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:14.366+0000 7f0250fae700 1 -- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 msgr2=0x7f0238040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1659741627 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:14.366+0000 7f0250fae700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 0x7f0238040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:15.167+0000 7f0250fae700 1 -- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 msgr2=0x7f0238040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1659741627 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:15.167+0000 7f0250fae700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 0x7f0238040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:16.768+0000 7f0250fae700 1 -- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 msgr2=0x7f0238040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1659741627 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:16.768+0000 7f0250fae700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 0x7f0238040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:18.456+0000 7f02427fc700 1 -- 172.21.15.122:0/946386765 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f023c01b440 con 0x7f024c07f710 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:18.456+0000 7f02427fc700 1 -- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 msgr2=0x7f0238040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:18.456+0000 7f02427fc700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 0x7f0238040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.457+0000 7f02427fc700 1 -- 172.21.15.122:0/946386765 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f023c02dd10 con 0x7f024c07f710 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.457+0000 7f02427fc700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0238041d00 0x7f02380440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.457+0000 7f02427fc700 1 -- 172.21.15.122:0/946386765 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f023c008e50 con 0x7f0238041d00 2024-05-03T17:45:20.405 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.457+0000 7f0250fae700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0238041d00 0x7f02380440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:20.406 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.458+0000 7f0250fae700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0238041d00 0x7f02380440f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0248003a10 tx=0x7f02480092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:20.406 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.459+0000 7f02427fc700 1 -- 172.21.15.122:0/946386765 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f023c008e50 con 0x7f0238041d00 2024-05-03T17:45:20.406 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.467+0000 7f0253a13700 1 -- 172.21.15.122:0/946386765 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f024c105620 con 0x7f0238041d00 2024-05-03T17:45:20.406 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.467+0000 7f02427fc700 1 -- 172.21.15.122:0/946386765 <== mgr.14152 v2:172.21.15.122:6800/1089873263 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f024c105620 con 0x7f0238041d00 2024-05-03T17:45:20.406 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.468+0000 7f0253a13700 1 -- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0238041d00 msgr2=0x7f02380440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:20.406 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.468+0000 7f0253a13700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0238041d00 0x7f02380440f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0248003a10 tx=0x7f02480092b0 comp rx=0 tx=0).stop 2024-05-03T17:45:20.406 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.468+0000 7f0253a13700 1 -- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 msgr2=0x7f024c19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:20.406 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.468+0000 7f0253a13700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c19ef40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f023c0116e0 tx=0x7f023c011aa0 comp rx=0 tx=0).stop 2024-05-03T17:45:20.406 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.468+0000 7f0253a13700 1 -- 172.21.15.122:0/946386765 shutdown_connections 2024-05-03T17:45:20.406 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.468+0000 7f0253a13700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0238041d00 0x7f02380440f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:20.406 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.468+0000 7f0253a13700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f024c07f710 0x7f024c19ef40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:20.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.468+0000 7f0253a13700 1 --2- 172.21.15.122:0/946386765 >> [v2:172.21.15.122:6800/1659741627,v1:172.21.15.122:6801/1659741627] conn(0x7f023803e470 0x7f0238040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:20.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.468+0000 7f0253a13700 1 -- 172.21.15.122:0/946386765 >> 172.21.15.122:0/946386765 conn(0x7f024c104410 msgr2=0x7f024c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:20.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.468+0000 7f0253a13700 1 -- 172.21.15.122:0/946386765 shutdown_connections 2024-05-03T17:45:20.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:19.468+0000 7f0253a13700 1 -- 172.21.15.122:0/946386765 wait complete. 2024-05-03T17:45:20.407 INFO:teuthology.orchestra.run.smithi122.stdout:mgr epoch 9 is available 2024-05-03T17:45:20.407 INFO:teuthology.orchestra.run.smithi122.stdout:Generating a dashboard self-signed certificate... 2024-05-03T17:45:20.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:20 smithi122 ceph-mon[29744]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-03T17:45:20.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:20 smithi122 ceph-mon[29744]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-03T17:45:20.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:20 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:21.755 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-03T17:45:21.755 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.884+0000 7fac68ad7700 1 Processor -- start 2024-05-03T17:45:21.755 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.884+0000 7fac68ad7700 1 -- start start 2024-05-03T17:45:21.755 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.884+0000 7fac68ad7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac64109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:21.756 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.884+0000 7fac68ad7700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac64109820 con 0x7fac64108e30 2024-05-03T17:45:21.756 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.885+0000 7fac6259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac64109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:21.756 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.885+0000 7fac6259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac64109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47664/0 (socket says 172.21.15.122:47664) 2024-05-03T17:45:21.756 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.885+0000 7fac6259c700 1 -- 172.21.15.122:0/1106023797 learned_addr learned my addr 172.21.15.122:0/1106023797 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:21.756 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.885+0000 7fac6259c700 1 -- 172.21.15.122:0/1106023797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac64109fe0 con 0x7fac64108e30 2024-05-03T17:45:21.756 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.885+0000 7fac6259c700 1 --2- 172.21.15.122:0/1106023797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac64109250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac540098a0 tx=0x7fac54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f372d25220d4cf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:21.756 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.885+0000 7fac6159a700 1 -- 172.21.15.122:0/1106023797 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac54004160 con 0x7fac64108e30 2024-05-03T17:45:21.756 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.885+0000 7fac6159a700 1 -- 172.21.15.122:0/1106023797 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac54004300 con 0x7fac64108e30 2024-05-03T17:45:21.756 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.885+0000 7fac6159a700 1 -- 172.21.15.122:0/1106023797 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac54011a20 con 0x7fac64108e30 2024-05-03T17:45:21.756 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.886+0000 7fac68ad7700 1 -- 172.21.15.122:0/1106023797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 msgr2=0x7fac64109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:21.756 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.886+0000 7fac68ad7700 1 --2- 172.21.15.122:0/1106023797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac64109250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac540098a0 tx=0x7fac54009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.886+0000 7fac68ad7700 1 -- 172.21.15.122:0/1106023797 shutdown_connections 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.886+0000 7fac68ad7700 1 --2- 172.21.15.122:0/1106023797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac64109250 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.886+0000 7fac68ad7700 1 -- 172.21.15.122:0/1106023797 >> 172.21.15.122:0/1106023797 conn(0x7fac64104410 msgr2=0x7fac64106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.886+0000 7fac68ad7700 1 -- 172.21.15.122:0/1106023797 shutdown_connections 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.886+0000 7fac68ad7700 1 -- 172.21.15.122:0/1106023797 wait complete. 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.887+0000 7fac68ad7700 1 Processor -- start 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.887+0000 7fac68ad7700 1 -- start start 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.887+0000 7fac68ad7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac6419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.887+0000 7fac68ad7700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac6410d390 con 0x7fac64108e30 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.887+0000 7fac6259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac6419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.887+0000 7fac6259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac6419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47678/0 (socket says 172.21.15.122:47678) 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.887+0000 7fac6259c700 1 -- 172.21.15.122:0/4196026487 learned_addr learned my addr 172.21.15.122:0/4196026487 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.888+0000 7fac6259c700 1 -- 172.21.15.122:0/4196026487 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6419f4a0 con 0x7fac64108e30 2024-05-03T17:45:21.757 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.888+0000 7fac6259c700 1 --2- 172.21.15.122:0/4196026487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac6419ef60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac540119f0 tx=0x7fac54011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.888+0000 7fac537fe700 1 -- 172.21.15.122:0/4196026487 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac540039e0 con 0x7fac64108e30 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.888+0000 7fac68ad7700 1 -- 172.21.15.122:0/4196026487 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6419f730 con 0x7fac64108e30 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.888+0000 7fac537fe700 1 -- 172.21.15.122:0/4196026487 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac5401b8f0 con 0x7fac64108e30 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.888+0000 7fac537fe700 1 -- 172.21.15.122:0/4196026487 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac540237a0 con 0x7fac64108e30 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.888+0000 7fac68ad7700 1 -- 172.21.15.122:0/4196026487 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac641a2420 con 0x7fac64108e30 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.889+0000 7fac537fe700 1 -- 172.21.15.122:0/4196026487 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fac5402c3f0 con 0x7fac64108e30 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.889+0000 7fac68ad7700 1 -- 172.21.15.122:0/4196026487 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6406a5c0 con 0x7fac64108e30 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.889+0000 7fac537fe700 1 --2- 172.21.15.122:0/4196026487 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac4c03e470 0x7fac4c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.889+0000 7fac537fe700 1 -- 172.21.15.122:0/4196026487 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fac54053f10 con 0x7fac64108e30 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.889+0000 7fac61d9b700 1 --2- 172.21.15.122:0/4196026487 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac4c03e470 0x7fac4c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.890+0000 7fac61d9b700 1 --2- 172.21.15.122:0/4196026487 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac4c03e470 0x7fac4c040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac58006fd0 tx=0x7fac58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:20.894+0000 7fac537fe700 1 -- 172.21.15.122:0/4196026487 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fac5401f030 con 0x7fac64108e30 2024-05-03T17:45:21.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.116+0000 7fac68ad7700 1 -- 172.21.15.122:0/4196026487 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fac64105580 con 0x7fac4c03e470 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.215+0000 7fac537fe700 1 -- 172.21.15.122:0/4196026487 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fac64105580 con 0x7fac4c03e470 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.217+0000 7fac68ad7700 1 -- 172.21.15.122:0/4196026487 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac4c03e470 msgr2=0x7fac4c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.217+0000 7fac68ad7700 1 --2- 172.21.15.122:0/4196026487 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac4c03e470 0x7fac4c040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac58006fd0 tx=0x7fac58006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.217+0000 7fac68ad7700 1 -- 172.21.15.122:0/4196026487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 msgr2=0x7fac6419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.217+0000 7fac68ad7700 1 --2- 172.21.15.122:0/4196026487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac6419ef60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac540119f0 tx=0x7fac54011ad0 comp rx=0 tx=0).stop 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.217+0000 7fac68ad7700 1 -- 172.21.15.122:0/4196026487 shutdown_connections 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.217+0000 7fac68ad7700 1 --2- 172.21.15.122:0/4196026487 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac4c03e470 0x7fac4c040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.217+0000 7fac68ad7700 1 --2- 172.21.15.122:0/4196026487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac64108e30 0x7fac6419ef60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.217+0000 7fac68ad7700 1 -- 172.21.15.122:0/4196026487 >> 172.21.15.122:0/4196026487 conn(0x7fac64104410 msgr2=0x7fac64104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.217+0000 7fac68ad7700 1 -- 172.21.15.122:0/4196026487 shutdown_connections 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:21.218+0000 7fac68ad7700 1 -- 172.21.15.122:0/4196026487 wait complete. 2024-05-03T17:45:21.759 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial admin user... 2024-05-03T17:45:21.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:21 smithi122 ceph-mon[29744]: mgrmap e12: x(active, since 2s) 2024-05-03T17:45:21.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:21.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:21 smithi122 ceph-mon[29744]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:45:21.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:21.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:22.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:22 smithi122 ceph-mon[29744]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:45:22.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:22 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:23.240 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$SQO9kuTbmp5MHNZAc0szf.cEMDrKzZCc/S63TEo/zK2lcizVmfkzu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714758322, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-03T17:45:23.240 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.166+0000 7f3fc5ef9700 1 Processor -- start 2024-05-03T17:45:23.240 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.166+0000 7f3fc5ef9700 1 -- start start 2024-05-03T17:45:23.240 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.166+0000 7f3fc5ef9700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc0109750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:23.240 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.166+0000 7f3fc5ef9700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc0109d20 con 0x7f3fc0109330 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.166+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc0109750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.167+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc0109750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47694/0 (socket says 172.21.15.122:47694) 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.167+0000 7f3fbf7fe700 1 -- 172.21.15.122:0/397184394 learned_addr learned my addr 172.21.15.122:0/397184394 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.167+0000 7f3fbf7fe700 1 -- 172.21.15.122:0/397184394 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc0107f40 con 0x7f3fc0109330 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.167+0000 7f3fbf7fe700 1 --2- 172.21.15.122:0/397184394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc0109750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00098a0 tx=0x7f3fb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2553215057a08a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.167+0000 7f3fbe7fc700 1 -- 172.21.15.122:0/397184394 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fb0004160 con 0x7f3fc0109330 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.167+0000 7f3fbe7fc700 1 -- 172.21.15.122:0/397184394 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fb0004300 con 0x7f3fc0109330 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.167+0000 7f3fbe7fc700 1 -- 172.21.15.122:0/397184394 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fb0011a20 con 0x7f3fc0109330 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.168+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/397184394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 msgr2=0x7f3fc0109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.168+0000 7f3fc5ef9700 1 --2- 172.21.15.122:0/397184394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc0109750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00098a0 tx=0x7f3fb0009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.168+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/397184394 shutdown_connections 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.168+0000 7f3fc5ef9700 1 --2- 172.21.15.122:0/397184394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc0109750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.168+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/397184394 >> 172.21.15.122:0/397184394 conn(0x7f3fc0102a70 msgr2=0x7f3fc0104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.168+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/397184394 shutdown_connections 2024-05-03T17:45:23.241 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.168+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/397184394 wait complete. 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.169+0000 7f3fc5ef9700 1 Processor -- start 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.169+0000 7f3fc5ef9700 1 -- start start 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.170+0000 7f3fc5ef9700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.170+0000 7f3fc5ef9700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc010c300 con 0x7f3fc0109330 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.170+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.170+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc019bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47706/0 (socket says 172.21.15.122:47706) 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.170+0000 7f3fbf7fe700 1 -- 172.21.15.122:0/3147302834 learned_addr learned my addr 172.21.15.122:0/3147302834 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.170+0000 7f3fbf7fe700 1 -- 172.21.15.122:0/3147302834 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc019c200 con 0x7f3fc0109330 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.170+0000 7f3fbf7fe700 1 --2- 172.21.15.122:0/3147302834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc019bcc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006af0 tx=0x7f3fb0003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.170+0000 7f3fbcff9700 1 -- 172.21.15.122:0/3147302834 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fb0011a20 con 0x7f3fc0109330 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.171+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/3147302834 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc019c490 con 0x7f3fc0109330 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.171+0000 7f3fbcff9700 1 -- 172.21.15.122:0/3147302834 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fb0003b10 con 0x7f3fc0109330 2024-05-03T17:45:23.242 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.171+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/3147302834 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc0106e30 con 0x7f3fc0109330 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.171+0000 7f3fbcff9700 1 -- 172.21.15.122:0/3147302834 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fb0023990 con 0x7f3fc0109330 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.172+0000 7f3fbcff9700 1 -- 172.21.15.122:0/3147302834 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f3fb0023b90 con 0x7f3fc0109330 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.172+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/3147302834 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc006a5c0 con 0x7f3fc0109330 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.172+0000 7f3fbcff9700 1 --2- 172.21.15.122:0/3147302834 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f3fa803e4d0 0x7f3fa8040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.172+0000 7f3fbcff9700 1 -- 172.21.15.122:0/3147302834 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3fb0055cc0 con 0x7f3fc0109330 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.172+0000 7f3fbeffd700 1 --2- 172.21.15.122:0/3147302834 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f3fa803e4d0 0x7f3fa8040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.172+0000 7f3fbeffd700 1 --2- 172.21.15.122:0/3147302834 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f3fa803e4d0 0x7f3fa8040990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4006fd0 tx=0x7f3fb4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.177+0000 7f3fbcff9700 1 -- 172.21.15.122:0/3147302834 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3fb001add0 con 0x7f3fc0109330 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.405+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/3147302834 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f3fc0103c20 con 0x7f3fa803e4d0 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.630+0000 7f3fbcff9700 1 -- 172.21.15.122:0/3147302834 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f3fc0103c20 con 0x7f3fa803e4d0 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.632+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/3147302834 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f3fa803e4d0 msgr2=0x7f3fa8040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.632+0000 7f3fc5ef9700 1 --2- 172.21.15.122:0/3147302834 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f3fa803e4d0 0x7f3fa8040990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4006fd0 tx=0x7f3fb4006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.632+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/3147302834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 msgr2=0x7f3fc019bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:23.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.632+0000 7f3fc5ef9700 1 --2- 172.21.15.122:0/3147302834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc019bcc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006af0 tx=0x7f3fb0003980 comp rx=0 tx=0).stop 2024-05-03T17:45:23.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.632+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/3147302834 shutdown_connections 2024-05-03T17:45:23.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.632+0000 7f3fc5ef9700 1 --2- 172.21.15.122:0/3147302834 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f3fa803e4d0 0x7f3fa8040990 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:23.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.632+0000 7f3fc5ef9700 1 --2- 172.21.15.122:0/3147302834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc0109330 0x7f3fc019bcc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:23.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.632+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/3147302834 >> 172.21.15.122:0/3147302834 conn(0x7f3fc0102a70 msgr2=0x7f3fc0103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:23.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.632+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/3147302834 shutdown_connections 2024-05-03T17:45:23.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:22.632+0000 7f3fc5ef9700 1 -- 172.21.15.122:0/3147302834 wait complete. 2024-05-03T17:45:23.244 INFO:teuthology.orchestra.run.smithi122.stdout:Fetching dashboard port number... 2024-05-03T17:45:24.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:24 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2035462013' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-03T17:45:24.799 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 8443 2024-05-03T17:45:24.799 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.735+0000 7f87a6155700 1 Processor -- start 2024-05-03T17:45:24.799 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.735+0000 7f87a6155700 1 -- start start 2024-05-03T17:45:24.799 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.735+0000 7f87a6155700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a010bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:24.799 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.735+0000 7f87a6155700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87a007ce00 con 0x7f87a0109980 2024-05-03T17:45:24.799 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.735+0000 7f879f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a010bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:24.799 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.735+0000 7f879f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a010bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47716/0 (socket says 172.21.15.122:47716) 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.735+0000 7f879f7fe700 1 -- 172.21.15.122:0/3285662733 learned_addr learned my addr 172.21.15.122:0/3285662733 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.736+0000 7f879f7fe700 1 -- 172.21.15.122:0/3285662733 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a010c2b0 con 0x7f87a0109980 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.736+0000 7f879f7fe700 1 --2- 172.21.15.122:0/3285662733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a010bd70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87900098a0 tx=0x7f8790009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15472dce1303ff06 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.736+0000 7f879e7fc700 1 -- 172.21.15.122:0/3285662733 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8790004160 con 0x7f87a0109980 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.736+0000 7f879e7fc700 1 -- 172.21.15.122:0/3285662733 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8790004300 con 0x7f87a0109980 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.736+0000 7f879e7fc700 1 -- 172.21.15.122:0/3285662733 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8790011a20 con 0x7f87a0109980 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.736+0000 7f87a6155700 1 -- 172.21.15.122:0/3285662733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 msgr2=0x7f87a010bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.736+0000 7f87a6155700 1 --2- 172.21.15.122:0/3285662733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a010bd70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87900098a0 tx=0x7f8790009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.736+0000 7f87a6155700 1 -- 172.21.15.122:0/3285662733 shutdown_connections 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.736+0000 7f87a6155700 1 --2- 172.21.15.122:0/3285662733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a010bd70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.736+0000 7f87a6155700 1 -- 172.21.15.122:0/3285662733 >> 172.21.15.122:0/3285662733 conn(0x7f87a01032f0 msgr2=0x7f87a0105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.737+0000 7f87a6155700 1 -- 172.21.15.122:0/3285662733 shutdown_connections 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.737+0000 7f87a6155700 1 -- 172.21.15.122:0/3285662733 wait complete. 2024-05-03T17:45:24.800 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.737+0000 7f87a6155700 1 Processor -- start 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.737+0000 7f87a6155700 1 -- start start 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.737+0000 7f87a6155700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a0109200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.737+0000 7f87a6155700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87a010d200 con 0x7f87a0109980 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.738+0000 7f879f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a0109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.738+0000 7f879f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a0109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47718/0 (socket says 172.21.15.122:47718) 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.738+0000 7f879f7fe700 1 -- 172.21.15.122:0/2035462013 learned_addr learned my addr 172.21.15.122:0/2035462013 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.738+0000 7f879f7fe700 1 -- 172.21.15.122:0/2035462013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a0109740 con 0x7f87a0109980 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.738+0000 7f879f7fe700 1 --2- 172.21.15.122:0/2035462013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a0109200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8790004660 tx=0x7f8790011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.738+0000 7f879cff9700 1 -- 172.21.15.122:0/2035462013 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8790003aa0 con 0x7f87a0109980 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.738+0000 7f87a6155700 1 -- 172.21.15.122:0/2035462013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a0107930 con 0x7f87a0109980 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.738+0000 7f879cff9700 1 -- 172.21.15.122:0/2035462013 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f879001b8f0 con 0x7f87a0109980 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.738+0000 7f879cff9700 1 -- 172.21.15.122:0/2035462013 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8790023900 con 0x7f87a0109980 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.738+0000 7f87a6155700 1 -- 172.21.15.122:0/2035462013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a0107e70 con 0x7f87a0109980 2024-05-03T17:45:24.801 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.739+0000 7f879cff9700 1 -- 172.21.15.122:0/2035462013 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f8790023bc0 con 0x7f87a0109980 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.739+0000 7f87a6155700 1 -- 172.21.15.122:0/2035462013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a006a5c0 con 0x7f87a0109980 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.739+0000 7f879cff9700 1 --2- 172.21.15.122:0/2035462013 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f878c03e4c0 0x7f878c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.739+0000 7f879cff9700 1 -- 172.21.15.122:0/2035462013 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8790053fb0 con 0x7f87a0109980 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.739+0000 7f879effd700 1 --2- 172.21.15.122:0/2035462013 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f878c03e4c0 0x7f878c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.740+0000 7f879effd700 1 --2- 172.21.15.122:0/2035462013 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f878c03e4c0 0x7f878c040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8788006fd0 tx=0x7f8788006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.744+0000 7f879cff9700 1 -- 172.21.15.122:0/2035462013 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f879001f030 con 0x7f87a0109980 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.951+0000 7f87a6155700 1 -- 172.21.15.122:0/2035462013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f87a01080f0 con 0x7f87a0109980 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.951+0000 7f879cff9700 1 -- 172.21.15.122:0/2035462013 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f87900202b0 con 0x7f87a0109980 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.953+0000 7f87a6155700 1 -- 172.21.15.122:0/2035462013 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f878c03e4c0 msgr2=0x7f878c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.953+0000 7f87a6155700 1 --2- 172.21.15.122:0/2035462013 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f878c03e4c0 0x7f878c040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8788006fd0 tx=0x7f8788006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.953+0000 7f87a6155700 1 -- 172.21.15.122:0/2035462013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 msgr2=0x7f87a0109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:24.802 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.953+0000 7f87a6155700 1 --2- 172.21.15.122:0/2035462013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a0109200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8790004660 tx=0x7f8790011aa0 comp rx=0 tx=0).stop 2024-05-03T17:45:24.803 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.954+0000 7f87a6155700 1 -- 172.21.15.122:0/2035462013 shutdown_connections 2024-05-03T17:45:24.803 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.954+0000 7f87a6155700 1 --2- 172.21.15.122:0/2035462013 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f878c03e4c0 0x7f878c040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:24.803 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.954+0000 7f87a6155700 1 --2- 172.21.15.122:0/2035462013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a0109980 0x7f87a0109200 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:24.803 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.954+0000 7f87a6155700 1 -- 172.21.15.122:0/2035462013 >> 172.21.15.122:0/2035462013 conn(0x7f87a01032f0 msgr2=0x7f87a0103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:24.803 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.954+0000 7f87a6155700 1 -- 172.21.15.122:0/2035462013 shutdown_connections 2024-05-03T17:45:24.803 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:23.954+0000 7f87a6155700 1 -- 172.21.15.122:0/2035462013 wait complete. 2024-05-03T17:45:24.806 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T17:45:24.806 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-05-03T17:45:24.813 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T17:45:24.813 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-05-03T17:45:24.814 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-05-03T17:45:24.814 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-03T17:45:24.815 INFO:teuthology.orchestra.run.smithi122.stdout:Ceph Dashboard is now available at: 2024-05-03T17:45:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: URL: https://smithi122.front.sepia.ceph.com:8443/ 2024-05-03T17:45:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: User: admin 2024-05-03T17:45:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: Password: 0vthxj1qu1 2024-05-03T17:45:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:24.816 INFO:teuthology.orchestra.run.smithi122.stdout:Saving cluster configuration to /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config directory 2024-05-03T17:45:24.816 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling autotune for osd_memory_target 2024-05-03T17:45:25.956 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.227+0000 7f9ffeee4700 1 Processor -- start 2024-05-03T17:45:25.956 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.227+0000 7f9ffeee4700 1 -- start start 2024-05-03T17:45:25.956 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.227+0000 7f9ffeee4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff810bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:25.956 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.227+0000 7f9ffeee4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff807ce20 con 0x7f9ff8109960 2024-05-03T17:45:25.956 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.227+0000 7f9ffcc80700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff810bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:25.956 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.227+0000 7f9ffcc80700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff810bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47728/0 (socket says 172.21.15.122:47728) 2024-05-03T17:45:25.956 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.227+0000 7f9ffcc80700 1 -- 172.21.15.122:0/1985139474 learned_addr learned my addr 172.21.15.122:0/1985139474 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:25.956 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.227+0000 7f9ffcc80700 1 -- 172.21.15.122:0/1985139474 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff810c290 con 0x7f9ff8109960 2024-05-03T17:45:25.956 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.227+0000 7f9ffcc80700 1 --2- 172.21.15.122:0/1985139474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff810bd50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0098a0 tx=0x7f9fec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6230ddf36418f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:25.956 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.228+0000 7f9ff77fe700 1 -- 172.21.15.122:0/1985139474 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fec004160 con 0x7f9ff8109960 2024-05-03T17:45:25.956 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.228+0000 7f9ff77fe700 1 -- 172.21.15.122:0/1985139474 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fec004300 con 0x7f9ff8109960 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.228+0000 7f9ff77fe700 1 -- 172.21.15.122:0/1985139474 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fec011a20 con 0x7f9ff8109960 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.228+0000 7f9ffeee4700 1 -- 172.21.15.122:0/1985139474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 msgr2=0x7f9ff810bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.228+0000 7f9ffeee4700 1 --2- 172.21.15.122:0/1985139474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff810bd50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0098a0 tx=0x7f9fec009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.228+0000 7f9ffeee4700 1 -- 172.21.15.122:0/1985139474 shutdown_connections 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.228+0000 7f9ffeee4700 1 --2- 172.21.15.122:0/1985139474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff810bd50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.228+0000 7f9ffeee4700 1 -- 172.21.15.122:0/1985139474 >> 172.21.15.122:0/1985139474 conn(0x7f9ff81032f0 msgr2=0x7f9ff8105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.228+0000 7f9ffeee4700 1 -- 172.21.15.122:0/1985139474 shutdown_connections 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.228+0000 7f9ffeee4700 1 -- 172.21.15.122:0/1985139474 wait complete. 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.229+0000 7f9ffeee4700 1 Processor -- start 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.229+0000 7f9ffeee4700 1 -- start start 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.229+0000 7f9ffeee4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff81091e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.229+0000 7f9ffeee4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff810d230 con 0x7f9ff8109960 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.229+0000 7f9ffcc80700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff81091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:25.957 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.229+0000 7f9ffcc80700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff81091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47738/0 (socket says 172.21.15.122:47738) 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.229+0000 7f9ffcc80700 1 -- 172.21.15.122:0/805933991 learned_addr learned my addr 172.21.15.122:0/805933991 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.229+0000 7f9ffcc80700 1 -- 172.21.15.122:0/805933991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff8109720 con 0x7f9ff8109960 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.229+0000 7f9ffcc80700 1 --2- 172.21.15.122:0/805933991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff81091e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0116e0 tx=0x7f9fec011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.230+0000 7f9ff5ffb700 1 -- 172.21.15.122:0/805933991 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fec003b20 con 0x7f9ff8109960 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.230+0000 7f9ff5ffb700 1 -- 172.21.15.122:0/805933991 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fec01b8f0 con 0x7f9ff8109960 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.230+0000 7f9ffeee4700 1 -- 172.21.15.122:0/805933991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff8107910 con 0x7f9ff8109960 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.230+0000 7f9ff5ffb700 1 -- 172.21.15.122:0/805933991 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fec0238c0 con 0x7f9ff8109960 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.230+0000 7f9ffeee4700 1 -- 172.21.15.122:0/805933991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff8107e50 con 0x7f9ff8109960 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.231+0000 7f9ff5ffb700 1 -- 172.21.15.122:0/805933991 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f9fec023b70 con 0x7f9ff8109960 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.231+0000 7f9ffeee4700 1 -- 172.21.15.122:0/805933991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff806a5c0 con 0x7f9ff8109960 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.231+0000 7f9ff5ffb700 1 --2- 172.21.15.122:0/805933991 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9fe803e510 0x7f9fe80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.231+0000 7f9ff5ffb700 1 -- 172.21.15.122:0/805933991 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9fec0543c0 con 0x7f9ff8109960 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.231+0000 7f9ff7fff700 1 --2- 172.21.15.122:0/805933991 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9fe803e510 0x7f9fe80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:25.958 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.231+0000 7f9ff7fff700 1 --2- 172.21.15.122:0/805933991 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9fe803e510 0x7f9fe80409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4006fd0 tx=0x7f9fe4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.236+0000 7f9ff5ffb700 1 -- 172.21.15.122:0/805933991 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9fec01f030 con 0x7f9ff8109960 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.435+0000 7f9ffeee4700 1 -- 172.21.15.122:0/805933991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f9ff81080d0 con 0x7f9ff8109960 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.435+0000 7f9ff5ffb700 1 -- 172.21.15.122:0/805933991 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f9fec0105e0 con 0x7f9ff8109960 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.438+0000 7f9ffeee4700 1 -- 172.21.15.122:0/805933991 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9fe803e510 msgr2=0x7f9fe80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.438+0000 7f9ffeee4700 1 --2- 172.21.15.122:0/805933991 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9fe803e510 0x7f9fe80409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4006fd0 tx=0x7f9fe4006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.438+0000 7f9ffeee4700 1 -- 172.21.15.122:0/805933991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 msgr2=0x7f9ff81091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.438+0000 7f9ffeee4700 1 --2- 172.21.15.122:0/805933991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff81091e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0116e0 tx=0x7f9fec011aa0 comp rx=0 tx=0).stop 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.438+0000 7f9ffeee4700 1 -- 172.21.15.122:0/805933991 shutdown_connections 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.438+0000 7f9ffeee4700 1 --2- 172.21.15.122:0/805933991 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9fe803e510 0x7f9fe80409d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.438+0000 7f9ffeee4700 1 --2- 172.21.15.122:0/805933991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff8109960 0x7f9ff81091e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.438+0000 7f9ffeee4700 1 -- 172.21.15.122:0/805933991 >> 172.21.15.122:0/805933991 conn(0x7f9ff81032f0 msgr2=0x7f9ff8103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.438+0000 7f9ffeee4700 1 -- 172.21.15.122:0/805933991 shutdown_connections 2024-05-03T17:45:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:25.438+0000 7f9ffeee4700 1 -- 172.21.15.122:0/805933991 wait complete. 2024-05-03T17:45:26.275 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:26 smithi122 ceph-mon[29744]: mgrmap e13: x(active, since 6s) 2024-05-03T17:45:27.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.379+0000 7f23a1d9e700 1 Processor -- start 2024-05-03T17:45:27.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.379+0000 7f23a1d9e700 1 -- start start 2024-05-03T17:45:27.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.379+0000 7f23a1d9e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c10bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:27.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.379+0000 7f23a1d9e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f239c07d6a0 con 0x7f239c109720 2024-05-03T17:45:27.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.379+0000 7f23a0d9c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c10bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:27.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.379+0000 7f23a0d9c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c10bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47742/0 (socket says 172.21.15.122:47742) 2024-05-03T17:45:27.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.379+0000 7f23a0d9c700 1 -- 172.21.15.122:0/1758324160 learned_addr learned my addr 172.21.15.122:0/1758324160 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:27.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.379+0000 7f23a0d9c700 1 -- 172.21.15.122:0/1758324160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c10c050 con 0x7f239c109720 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.379+0000 7f23a0d9c700 1 --2- 172.21.15.122:0/1758324160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c10bb10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f23900098a0 tx=0x7f2390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfb7f197cbf8986 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.380+0000 7f239b7fe700 1 -- 172.21.15.122:0/1758324160 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2390004160 con 0x7f239c109720 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.380+0000 7f239b7fe700 1 -- 172.21.15.122:0/1758324160 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2390004300 con 0x7f239c109720 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.380+0000 7f239b7fe700 1 -- 172.21.15.122:0/1758324160 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2390011a20 con 0x7f239c109720 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.380+0000 7f23a1d9e700 1 -- 172.21.15.122:0/1758324160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 msgr2=0x7f239c10bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.380+0000 7f23a1d9e700 1 --2- 172.21.15.122:0/1758324160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c10bb10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f23900098a0 tx=0x7f2390009bb0 comp rx=0 tx=0).stop 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.380+0000 7f23a1d9e700 1 -- 172.21.15.122:0/1758324160 shutdown_connections 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.380+0000 7f23a1d9e700 1 --2- 172.21.15.122:0/1758324160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c10bb10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.380+0000 7f23a1d9e700 1 -- 172.21.15.122:0/1758324160 >> 172.21.15.122:0/1758324160 conn(0x7f239c103130 msgr2=0x7f239c105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.380+0000 7f23a1d9e700 1 -- 172.21.15.122:0/1758324160 shutdown_connections 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.380+0000 7f23a1d9e700 1 -- 172.21.15.122:0/1758324160 wait complete. 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.381+0000 7f23a1d9e700 1 Processor -- start 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.381+0000 7f23a1d9e700 1 -- start start 2024-05-03T17:45:27.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.381+0000 7f23a1d9e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:27.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.381+0000 7f23a1d9e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f239c10cff0 con 0x7f239c109720 2024-05-03T17:45:27.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.381+0000 7f23a0d9c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:27.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.381+0000 7f23a0d9c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c19cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47744/0 (socket says 172.21.15.122:47744) 2024-05-03T17:45:27.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.381+0000 7f23a0d9c700 1 -- 172.21.15.122:0/275849888 learned_addr learned my addr 172.21.15.122:0/275849888 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:27.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.382+0000 7f23a0d9c700 1 -- 172.21.15.122:0/275849888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c19d070 con 0x7f239c109720 2024-05-03T17:45:27.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.382+0000 7f23a0d9c700 1 --2- 172.21.15.122:0/275849888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c19cb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23900116e0 tx=0x7f2390011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:27.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.382+0000 7f2399ffb700 1 -- 172.21.15.122:0/275849888 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2390003b20 con 0x7f239c109720 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.382+0000 7f23a1d9e700 1 -- 172.21.15.122:0/275849888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239c19d300 con 0x7f239c109720 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.382+0000 7f2399ffb700 1 -- 172.21.15.122:0/275849888 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f239001b8f0 con 0x7f239c109720 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.382+0000 7f2399ffb700 1 -- 172.21.15.122:0/275849888 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23900238c0 con 0x7f239c109720 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.382+0000 7f23a1d9e700 1 -- 172.21.15.122:0/275849888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239c19fff0 con 0x7f239c109720 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.383+0000 7f2399ffb700 1 -- 172.21.15.122:0/275849888 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f239001baa0 con 0x7f239c109720 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.383+0000 7f23a1d9e700 1 -- 172.21.15.122:0/275849888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f239c06a5c0 con 0x7f239c109720 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.383+0000 7f2399ffb700 1 --2- 172.21.15.122:0/275849888 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f238c03e510 0x7f238c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.383+0000 7f2399ffb700 1 -- 172.21.15.122:0/275849888 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f23900541b0 con 0x7f239c109720 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.383+0000 7f239bfff700 1 --2- 172.21.15.122:0/275849888 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f238c03e510 0x7f238c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.384+0000 7f239bfff700 1 --2- 172.21.15.122:0/275849888 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f238c03e510 0x7f238c0409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2388006fd0 tx=0x7f2388006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.388+0000 7f2399ffb700 1 -- 172.21.15.122:0/275849888 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f239001f030 con 0x7f239c109720 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.702+0000 7f23a1d9e700 1 -- 172.21.15.122:0/275849888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f239c1a05a0 con 0x7f239c109720 2024-05-03T17:45:27.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.704+0000 7f2399ffb700 1 -- 172.21.15.122:0/275849888 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f2390018590 con 0x7f239c109720 2024-05-03T17:45:27.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.706+0000 7f23a1d9e700 1 -- 172.21.15.122:0/275849888 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f238c03e510 msgr2=0x7f238c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:27.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.706+0000 7f23a1d9e700 1 --2- 172.21.15.122:0/275849888 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f238c03e510 0x7f238c0409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2388006fd0 tx=0x7f2388006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:27.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.706+0000 7f23a1d9e700 1 -- 172.21.15.122:0/275849888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 msgr2=0x7f239c19cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:27.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.706+0000 7f23a1d9e700 1 --2- 172.21.15.122:0/275849888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c19cb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23900116e0 tx=0x7f2390011aa0 comp rx=0 tx=0).stop 2024-05-03T17:45:27.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.707+0000 7f23a1d9e700 1 -- 172.21.15.122:0/275849888 shutdown_connections 2024-05-03T17:45:27.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.707+0000 7f23a1d9e700 1 --2- 172.21.15.122:0/275849888 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f238c03e510 0x7f238c0409d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:27.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.707+0000 7f23a1d9e700 1 --2- 172.21.15.122:0/275849888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f239c109720 0x7f239c19cb30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:27.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.707+0000 7f23a1d9e700 1 -- 172.21.15.122:0/275849888 >> 172.21.15.122:0/275849888 conn(0x7f239c103130 msgr2=0x7f239c103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:27.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.707+0000 7f23a1d9e700 1 -- 172.21.15.122:0/275849888 shutdown_connections 2024-05-03T17:45:27.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-05-03T17:45:26.707+0000 7f23a1d9e700 1 -- 172.21.15.122:0/275849888 wait complete. 2024-05-03T17:45:27.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-03T17:45:27.200 INFO:teuthology.orchestra.run.smithi122.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-03T17:45:27.200 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:27.200 INFO:teuthology.orchestra.run.smithi122.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-03T17:45:27.200 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:27.200 INFO:teuthology.orchestra.run.smithi122.stdout:Or, if you are only running a single cluster on this host: 2024-05-03T17:45:27.200 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:27.200 INFO:teuthology.orchestra.run.smithi122.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-03T17:45:27.201 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:27.201 INFO:teuthology.orchestra.run.smithi122.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-03T17:45:27.201 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:27.201 INFO:teuthology.orchestra.run.smithi122.stdout: ceph telemetry on 2024-05-03T17:45:27.201 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:27.201 INFO:teuthology.orchestra.run.smithi122.stdout:For more information see: 2024-05-03T17:45:27.201 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:27.202 INFO:teuthology.orchestra.run.smithi122.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-03T17:45:27.202 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:27.202 INFO:teuthology.orchestra.run.smithi122.stdout:Bootstrap complete. 2024-05-03T17:45:27.229 INFO:tasks.cephadm:Fetching config... 2024-05-03T17:45:27.229 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:45:27.229 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-03T17:45:27.244 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-03T17:45:27.245 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:45:27.245 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-03T17:45:27.299 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-03T17:45:27.299 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:45:27.300 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-03T17:45:27.364 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-03T17:45:27.364 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:45:27.364 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-03T17:45:27.419 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-03T17:45:27.419 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMLpBLSSoKkrRndKn/089lrnZFCLNv4yBbaog3HZOhWegepqWW2M1sbkMSu3YTqUau57wAmFUzE7IFQyWD0QkDCgLJqsaHMOYNTuLU8ToTNFL77gshDG2Qn+rU7oPqjdZF09lhtaoljTEKHymaopi/QwEBb+VsG0FfyrFIaUBedE3EL2Ibv8OFSPQe4nkltiU9eYmHbygEKRZFJhElhc53Qh2cZrbMNmcxc7iTAr3OYU/Pmra7diLth/p1iuF74IuGt+Zbh4SdlSjj99U7jQ+6QLBL8+1exU45bi6deNtLP76CuvHHM8F1Ki8g5cITlnR/CU8XerpJimuI/nE1nw8ejnmPleeDSWx1+RKtJxi0ICZ1+WnV3OqepZmt1COhWReQPWMPixDQ+YOpmY9GR0grogBaxvFSmN57INwFbL/dUW+dfEDn1+1NBRKisYjba/2t8bND9bKoVM2k/sUTI9VAa0jGmdXdmFGr4AUEbvi31/mt5vh6IOOFfCEWQwh4Yuc= ceph-a46fc846-0974-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T17:45:27.490 INFO:teuthology.orchestra.run.smithi122.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMLpBLSSoKkrRndKn/089lrnZFCLNv4yBbaog3HZOhWegepqWW2M1sbkMSu3YTqUau57wAmFUzE7IFQyWD0QkDCgLJqsaHMOYNTuLU8ToTNFL77gshDG2Qn+rU7oPqjdZF09lhtaoljTEKHymaopi/QwEBb+VsG0FfyrFIaUBedE3EL2Ibv8OFSPQe4nkltiU9eYmHbygEKRZFJhElhc53Qh2cZrbMNmcxc7iTAr3OYU/Pmra7diLth/p1iuF74IuGt+Zbh4SdlSjj99U7jQ+6QLBL8+1exU45bi6deNtLP76CuvHHM8F1Ki8g5cITlnR/CU8XerpJimuI/nE1nw8ejnmPleeDSWx1+RKtJxi0ICZ1+WnV3OqepZmt1COhWReQPWMPixDQ+YOpmY9GR0grogBaxvFSmN57INwFbL/dUW+dfEDn1+1NBRKisYjba/2t8bND9bKoVM2k/sUTI9VAa0jGmdXdmFGr4AUEbvi31/mt5vh6IOOFfCEWQwh4Yuc= ceph-a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:45:27.512 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMLpBLSSoKkrRndKn/089lrnZFCLNv4yBbaog3HZOhWegepqWW2M1sbkMSu3YTqUau57wAmFUzE7IFQyWD0QkDCgLJqsaHMOYNTuLU8ToTNFL77gshDG2Qn+rU7oPqjdZF09lhtaoljTEKHymaopi/QwEBb+VsG0FfyrFIaUBedE3EL2Ibv8OFSPQe4nkltiU9eYmHbygEKRZFJhElhc53Qh2cZrbMNmcxc7iTAr3OYU/Pmra7diLth/p1iuF74IuGt+Zbh4SdlSjj99U7jQ+6QLBL8+1exU45bi6deNtLP76CuvHHM8F1Ki8g5cITlnR/CU8XerpJimuI/nE1nw8ejnmPleeDSWx1+RKtJxi0ICZ1+WnV3OqepZmt1COhWReQPWMPixDQ+YOpmY9GR0grogBaxvFSmN57INwFbL/dUW+dfEDn1+1NBRKisYjba/2t8bND9bKoVM2k/sUTI9VAa0jGmdXdmFGr4AUEbvi31/mt5vh6IOOFfCEWQwh4Yuc= ceph-a46fc846-0974-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T17:45:27.551 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMLpBLSSoKkrRndKn/089lrnZFCLNv4yBbaog3HZOhWegepqWW2M1sbkMSu3YTqUau57wAmFUzE7IFQyWD0QkDCgLJqsaHMOYNTuLU8ToTNFL77gshDG2Qn+rU7oPqjdZF09lhtaoljTEKHymaopi/QwEBb+VsG0FfyrFIaUBedE3EL2Ibv8OFSPQe4nkltiU9eYmHbygEKRZFJhElhc53Qh2cZrbMNmcxc7iTAr3OYU/Pmra7diLth/p1iuF74IuGt+Zbh4SdlSjj99U7jQ+6QLBL8+1exU45bi6deNtLP76CuvHHM8F1Ki8g5cITlnR/CU8XerpJimuI/nE1nw8ejnmPleeDSWx1+RKtJxi0ICZ1+WnV3OqepZmt1COhWReQPWMPixDQ+YOpmY9GR0grogBaxvFSmN57INwFbL/dUW+dfEDn1+1NBRKisYjba/2t8bND9bKoVM2k/sUTI9VAa0jGmdXdmFGr4AUEbvi31/mt5vh6IOOFfCEWQwh4Yuc= ceph-a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:45:27.563 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMLpBLSSoKkrRndKn/089lrnZFCLNv4yBbaog3HZOhWegepqWW2M1sbkMSu3YTqUau57wAmFUzE7IFQyWD0QkDCgLJqsaHMOYNTuLU8ToTNFL77gshDG2Qn+rU7oPqjdZF09lhtaoljTEKHymaopi/QwEBb+VsG0FfyrFIaUBedE3EL2Ibv8OFSPQe4nkltiU9eYmHbygEKRZFJhElhc53Qh2cZrbMNmcxc7iTAr3OYU/Pmra7diLth/p1iuF74IuGt+Zbh4SdlSjj99U7jQ+6QLBL8+1exU45bi6deNtLP76CuvHHM8F1Ki8g5cITlnR/CU8XerpJimuI/nE1nw8ejnmPleeDSWx1+RKtJxi0ICZ1+WnV3OqepZmt1COhWReQPWMPixDQ+YOpmY9GR0grogBaxvFSmN57INwFbL/dUW+dfEDn1+1NBRKisYjba/2t8bND9bKoVM2k/sUTI9VAa0jGmdXdmFGr4AUEbvi31/mt5vh6IOOFfCEWQwh4Yuc= ceph-a46fc846-0974-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T17:45:27.603 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMLpBLSSoKkrRndKn/089lrnZFCLNv4yBbaog3HZOhWegepqWW2M1sbkMSu3YTqUau57wAmFUzE7IFQyWD0QkDCgLJqsaHMOYNTuLU8ToTNFL77gshDG2Qn+rU7oPqjdZF09lhtaoljTEKHymaopi/QwEBb+VsG0FfyrFIaUBedE3EL2Ibv8OFSPQe4nkltiU9eYmHbygEKRZFJhElhc53Qh2cZrbMNmcxc7iTAr3OYU/Pmra7diLth/p1iuF74IuGt+Zbh4SdlSjj99U7jQ+6QLBL8+1exU45bi6deNtLP76CuvHHM8F1Ki8g5cITlnR/CU8XerpJimuI/nE1nw8ejnmPleeDSWx1+RKtJxi0ICZ1+WnV3OqepZmt1COhWReQPWMPixDQ+YOpmY9GR0grogBaxvFSmN57INwFbL/dUW+dfEDn1+1NBRKisYjba/2t8bND9bKoVM2k/sUTI9VAa0jGmdXdmFGr4AUEbvi31/mt5vh6IOOFfCEWQwh4Yuc= ceph-a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:45:27.614 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-03T17:45:27.848 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:45:27.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:27 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/275849888' entity='client.admin' 2024-05-03T17:45:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.119+0000 7f0c0aee1700 1 -- 172.21.15.122:0/2817300467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c04102ea0 msgr2=0x7f0c041032c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.119+0000 7f0c0aee1700 1 --2- 172.21.15.122:0/2817300467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c04102ea0 0x7f0c041032c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400b0a0 tx=0x7f0bf400b3b0 comp rx=0 tx=0).stop 2024-05-03T17:45:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.119+0000 7f0c0aee1700 1 -- 172.21.15.122:0/2817300467 shutdown_connections 2024-05-03T17:45:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.119+0000 7f0c0aee1700 1 --2- 172.21.15.122:0/2817300467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c04102ea0 0x7f0c041032c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.119+0000 7f0c0aee1700 1 -- 172.21.15.122:0/2817300467 >> 172.21.15.122:0/2817300467 conn(0x7f0c040fe490 msgr2=0x7f0c041008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.119+0000 7f0c0aee1700 1 -- 172.21.15.122:0/2817300467 shutdown_connections 2024-05-03T17:45:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.119+0000 7f0c0aee1700 1 -- 172.21.15.122:0/2817300467 wait complete. 2024-05-03T17:45:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.119+0000 7f0c0aee1700 1 Processor -- start 2024-05-03T17:45:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.119+0000 7f0c0aee1700 1 -- start start 2024-05-03T17:45:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.120+0000 7f0c0aee1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c04102ea0 0x7f0c041984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.120+0000 7f0c0aee1700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04108da0 con 0x7f0c04102ea0 2024-05-03T17:45:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.120+0000 7f0c08c7d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c04102ea0 0x7f0c041984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.120+0000 7f0c08c7d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c04102ea0 0x7f0c041984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47166/0 (socket says 172.21.15.122:47166) 2024-05-03T17:45:30.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.120+0000 7f0c08c7d700 1 -- 172.21.15.122:0/3814132900 learned_addr learned my addr 172.21.15.122:0/3814132900 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:30.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.120+0000 7f0c08c7d700 1 -- 172.21.15.122:0/3814132900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c04198a10 con 0x7f0c04102ea0 2024-05-03T17:45:30.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.120+0000 7f0c08c7d700 1 --2- 172.21.15.122:0/3814132900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c04102ea0 0x7f0c041984d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4007730 tx=0x7f0bf4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:30.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.120+0000 7f0c01ffb700 1 -- 172.21.15.122:0/3814132900 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bf4021020 con 0x7f0c04102ea0 2024-05-03T17:45:30.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.120+0000 7f0c0aee1700 1 -- 172.21.15.122:0/3814132900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c04198ca0 con 0x7f0c04102ea0 2024-05-03T17:45:30.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.120+0000 7f0c01ffb700 1 -- 172.21.15.122:0/3814132900 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf40044b0 con 0x7f0c04102ea0 2024-05-03T17:45:30.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.120+0000 7f0c01ffb700 1 -- 172.21.15.122:0/3814132900 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bf4017040 con 0x7f0c04102ea0 2024-05-03T17:45:30.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.121+0000 7f0c0aee1700 1 -- 172.21.15.122:0/3814132900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0419b990 con 0x7f0c04102ea0 2024-05-03T17:45:30.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.121+0000 7f0c01ffb700 1 -- 172.21.15.122:0/3814132900 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f0bf4024020 con 0x7f0c04102ea0 2024-05-03T17:45:30.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.121+0000 7f0c0aee1700 1 -- 172.21.15.122:0/3814132900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c04109520 con 0x7f0c04102ea0 2024-05-03T17:45:30.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.121+0000 7f0c01ffb700 1 --2- 172.21.15.122:0/3814132900 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0bf003e560 0x7f0bf0040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:30.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.122+0000 7f0c01ffb700 1 -- 172.21.15.122:0/3814132900 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0bf400f050 con 0x7f0c04102ea0 2024-05-03T17:45:30.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.122+0000 7f0c03fff700 1 --2- 172.21.15.122:0/3814132900 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0bf003e560 0x7f0bf0040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:30.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.122+0000 7f0c03fff700 1 --2- 172.21.15.122:0/3814132900 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0bf003e560 0x7f0bf0040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0bec006fd0 tx=0x7f0bec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:30.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.127+0000 7f0c01ffb700 1 -- 172.21.15.122:0/3814132900 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0bf402b020 con 0x7f0c04102ea0 2024-05-03T17:45:30.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.328+0000 7f0c0aee1700 1 -- 172.21.15.122:0/3814132900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f0c0402e580 con 0x7f0c04102ea0 2024-05-03T17:45:30.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.330+0000 7f0c01ffb700 1 -- 172.21.15.122:0/3814132900 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f0bf401ca70 con 0x7f0c04102ea0 2024-05-03T17:45:30.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.333+0000 7f0c0aee1700 1 -- 172.21.15.122:0/3814132900 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0bf003e560 msgr2=0x7f0bf0040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:30.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.333+0000 7f0c0aee1700 1 --2- 172.21.15.122:0/3814132900 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0bf003e560 0x7f0bf0040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0bec006fd0 tx=0x7f0bec006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:30.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.333+0000 7f0c0aee1700 1 -- 172.21.15.122:0/3814132900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c04102ea0 msgr2=0x7f0c041984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:30.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.333+0000 7f0c0aee1700 1 --2- 172.21.15.122:0/3814132900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c04102ea0 0x7f0c041984d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4007730 tx=0x7f0bf4003ce0 comp rx=0 tx=0).stop 2024-05-03T17:45:30.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.333+0000 7f0c0aee1700 1 -- 172.21.15.122:0/3814132900 shutdown_connections 2024-05-03T17:45:30.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.333+0000 7f0c0aee1700 1 --2- 172.21.15.122:0/3814132900 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0bf003e560 0x7f0bf0040a20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:30.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.333+0000 7f0c0aee1700 1 --2- 172.21.15.122:0/3814132900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c04102ea0 0x7f0c041984d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:30.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.333+0000 7f0c0aee1700 1 -- 172.21.15.122:0/3814132900 >> 172.21.15.122:0/3814132900 conn(0x7f0c040fe490 msgr2=0x7f0c040fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:30.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.333+0000 7f0c0aee1700 1 -- 172.21.15.122:0/3814132900 shutdown_connections 2024-05-03T17:45:30.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:30.333+0000 7f0c0aee1700 1 -- 172.21.15.122:0/3814132900 wait complete. 2024-05-03T17:45:30.900 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:30.900 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:30.900 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:45:30.900 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:30.900 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:45:30.900 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:30.900 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:30 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3814132900' entity='client.admin' 2024-05-03T17:45:30.900 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:45:30.900 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:30.926 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-03T17:45:30.927 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-03T17:45:31.159 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:45:32.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.518+0000 7efbffb0b700 1 -- 172.21.15.122:0/3375975309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efbf8101df0 msgr2=0x7efbf8102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.518+0000 7efbffb0b700 1 --2- 172.21.15.122:0/3375975309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efbf8101df0 0x7efbf8102210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efbf400b0a0 tx=0x7efbf400b3b0 comp rx=0 tx=0).stop 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.518+0000 7efbffb0b700 1 -- 172.21.15.122:0/3375975309 shutdown_connections 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.518+0000 7efbffb0b700 1 --2- 172.21.15.122:0/3375975309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efbf8101df0 0x7efbf8102210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.518+0000 7efbffb0b700 1 -- 172.21.15.122:0/3375975309 >> 172.21.15.122:0/3375975309 conn(0x7efbf80fd3b0 msgr2=0x7efbf80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.518+0000 7efbffb0b700 1 -- 172.21.15.122:0/3375975309 shutdown_connections 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.518+0000 7efbffb0b700 1 -- 172.21.15.122:0/3375975309 wait complete. 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbffb0b700 1 Processor -- start 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbffb0b700 1 -- start start 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbffb0b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efbf8101df0 0x7efbf806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbffb0b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8107d30 con 0x7efbf8101df0 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbfd8a7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efbf8101df0 0x7efbf806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:32.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbfd8a7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efbf8101df0 0x7efbf806bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47186/0 (socket says 172.21.15.122:47186) 2024-05-03T17:45:32.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbfd8a7700 1 -- 172.21.15.122:0/3367707937 learned_addr learned my addr 172.21.15.122:0/3367707937 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:32.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbfd8a7700 1 -- 172.21.15.122:0/3367707937 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf819b200 con 0x7efbf8101df0 2024-05-03T17:45:32.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbfd8a7700 1 --2- 172.21.15.122:0/3367707937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efbf8101df0 0x7efbf806bbe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efbf400d010 tx=0x7efbf4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:32.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbeeffd700 1 -- 172.21.15.122:0/3367707937 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf4021020 con 0x7efbf8101df0 2024-05-03T17:45:32.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbffb0b700 1 -- 172.21.15.122:0/3367707937 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf819b3d0 con 0x7efbf8101df0 2024-05-03T17:45:32.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbeeffd700 1 -- 172.21.15.122:0/3367707937 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf400bdd0 con 0x7efbf8101df0 2024-05-03T17:45:32.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.519+0000 7efbeeffd700 1 -- 172.21.15.122:0/3367707937 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf4017040 con 0x7efbf8101df0 2024-05-03T17:45:32.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.520+0000 7efbffb0b700 1 -- 172.21.15.122:0/3367707937 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf819b770 con 0x7efbf8101df0 2024-05-03T17:45:32.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.520+0000 7efbffb0b700 1 -- 172.21.15.122:0/3367707937 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf81084b0 con 0x7efbf8101df0 2024-05-03T17:45:32.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.521+0000 7efbeeffd700 1 -- 172.21.15.122:0/3367707937 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7efbf4024020 con 0x7efbf8101df0 2024-05-03T17:45:32.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.521+0000 7efbeeffd700 1 --2- 172.21.15.122:0/3367707937 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efbe403e510 0x7efbe40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:32.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.521+0000 7efbeeffd700 1 -- 172.21.15.122:0/3367707937 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efbf400f050 con 0x7efbf8101df0 2024-05-03T17:45:32.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.521+0000 7efbfd0a6700 1 --2- 172.21.15.122:0/3367707937 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efbe403e510 0x7efbe40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:32.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.521+0000 7efbfd0a6700 1 --2- 172.21.15.122:0/3367707937 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efbe403e510 0x7efbe40409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efbe8006fd0 tx=0x7efbe8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:32.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.526+0000 7efbeeffd700 1 -- 172.21.15.122:0/3367707937 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efbf402b020 con 0x7efbf8101df0 2024-05-03T17:45:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.733+0000 7efbffb0b700 1 -- 172.21.15.122:0/3367707937 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7efbf802e620 con 0x7efbe403e510 2024-05-03T17:45:32.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.736+0000 7efbeeffd700 1 -- 172.21.15.122:0/3367707937 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efbf802e620 con 0x7efbe403e510 2024-05-03T17:45:32.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.738+0000 7efbffb0b700 1 -- 172.21.15.122:0/3367707937 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efbe403e510 msgr2=0x7efbe40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:32.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.738+0000 7efbffb0b700 1 --2- 172.21.15.122:0/3367707937 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efbe403e510 0x7efbe40409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efbe8006fd0 tx=0x7efbe8006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:32.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.738+0000 7efbffb0b700 1 -- 172.21.15.122:0/3367707937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efbf8101df0 msgr2=0x7efbf806bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:32.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.738+0000 7efbffb0b700 1 --2- 172.21.15.122:0/3367707937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efbf8101df0 0x7efbf806bbe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efbf400d010 tx=0x7efbf4003ce0 comp rx=0 tx=0).stop 2024-05-03T17:45:32.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.738+0000 7efbffb0b700 1 -- 172.21.15.122:0/3367707937 shutdown_connections 2024-05-03T17:45:32.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.738+0000 7efbffb0b700 1 --2- 172.21.15.122:0/3367707937 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efbe403e510 0x7efbe40409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:32.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.738+0000 7efbffb0b700 1 --2- 172.21.15.122:0/3367707937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efbf8101df0 0x7efbf806bbe0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:32.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.738+0000 7efbffb0b700 1 -- 172.21.15.122:0/3367707937 >> 172.21.15.122:0/3367707937 conn(0x7efbf80fd3b0 msgr2=0x7efbf80fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:32.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.739+0000 7efbffb0b700 1 -- 172.21.15.122:0/3367707937 shutdown_connections 2024-05-03T17:45:32.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:32.739+0000 7efbffb0b700 1 -- 172.21.15.122:0/3367707937 wait complete. 2024-05-03T17:45:33.460 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-05-03T17:45:33.460 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T17:45:33.460 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-05-03T17:45:33.479 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T17:45:33.479 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:45:33.537 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-05-03T17:45:33.537 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch host add smithi149 2024-05-03T17:45:33.778 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:45:33.998 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:33 smithi122 ceph-mon[29744]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:45:33.998 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:33.998 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:45:33.998 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:45:33.998 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:45:33.998 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:33 smithi122 ceph-mon[29744]: Updating smithi122:/etc/ceph/ceph.conf 2024-05-03T17:45:33.999 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:33 smithi122 ceph-mon[29744]: Updating smithi122:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:45:33.999 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:33 smithi122 ceph-mon[29744]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:45:33.999 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:33.999 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:33.999 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:34.936 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:34 smithi122 ceph-mon[29744]: Updating smithi122:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T17:45:35.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.147+0000 7fae52343700 1 -- 172.21.15.122:0/3299443033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4c101fe0 msgr2=0x7fae4c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:35.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.147+0000 7fae52343700 1 --2- 172.21.15.122:0/3299443033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4c101fe0 0x7fae4c1023c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009910 tx=0x7fae3c009c20 comp rx=0 tx=0).stop 2024-05-03T17:45:35.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.147+0000 7fae52343700 1 -- 172.21.15.122:0/3299443033 shutdown_connections 2024-05-03T17:45:35.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.147+0000 7fae52343700 1 --2- 172.21.15.122:0/3299443033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4c101fe0 0x7fae4c1023c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:35.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.147+0000 7fae52343700 1 -- 172.21.15.122:0/3299443033 >> 172.21.15.122:0/3299443033 conn(0x7fae4c078a70 msgr2=0x7fae4c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:35.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.147+0000 7fae52343700 1 -- 172.21.15.122:0/3299443033 shutdown_connections 2024-05-03T17:45:35.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.147+0000 7fae52343700 1 -- 172.21.15.122:0/3299443033 wait complete. 2024-05-03T17:45:35.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.148+0000 7fae52343700 1 Processor -- start 2024-05-03T17:45:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.148+0000 7fae52343700 1 -- start start 2024-05-03T17:45:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.148+0000 7fae52343700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4c101fe0 0x7fae4c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.148+0000 7fae52343700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c10c260 con 0x7fae4c101fe0 2024-05-03T17:45:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.148+0000 7fae4bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4c101fe0 0x7fae4c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.148+0000 7fae4bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4c101fe0 0x7fae4c199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47200/0 (socket says 172.21.15.122:47200) 2024-05-03T17:45:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.148+0000 7fae4bfff700 1 -- 172.21.15.122:0/3704163187 learned_addr learned my addr 172.21.15.122:0/3704163187 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.148+0000 7fae4bfff700 1 -- 172.21.15.122:0/3704163187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c199c60 con 0x7fae4c101fe0 2024-05-03T17:45:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.148+0000 7fae4bfff700 1 --2- 172.21.15.122:0/3704163187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4c101fe0 0x7fae4c199720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae3c006010 tx=0x7fae3c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.149+0000 7fae497fa700 1 -- 172.21.15.122:0/3704163187 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3c026070 con 0x7fae4c101fe0 2024-05-03T17:45:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.149+0000 7fae497fa700 1 -- 172.21.15.122:0/3704163187 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae3c00bba0 con 0x7fae4c101fe0 2024-05-03T17:45:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.149+0000 7fae52343700 1 -- 172.21.15.122:0/3704163187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4c199f50 con 0x7fae4c101fe0 2024-05-03T17:45:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.149+0000 7fae497fa700 1 -- 172.21.15.122:0/3704163187 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3c003910 con 0x7fae4c101fe0 2024-05-03T17:45:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.149+0000 7fae52343700 1 -- 172.21.15.122:0/3704163187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4c19a410 con 0x7fae4c101fe0 2024-05-03T17:45:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.149+0000 7fae497fa700 1 -- 172.21.15.122:0/3704163187 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fae3c029070 con 0x7fae4c101fe0 2024-05-03T17:45:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.150+0000 7fae52343700 1 -- 172.21.15.122:0/3704163187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4c1034c0 con 0x7fae4c101fe0 2024-05-03T17:45:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.150+0000 7fae497fa700 1 --2- 172.21.15.122:0/3704163187 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fae3403e510 0x7fae340409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.150+0000 7fae497fa700 1 -- 172.21.15.122:0/3704163187 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fae3c01e070 con 0x7fae4c101fe0 2024-05-03T17:45:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.150+0000 7fae4b7fe700 1 --2- 172.21.15.122:0/3704163187 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fae3403e510 0x7fae340409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.150+0000 7fae4b7fe700 1 --2- 172.21.15.122:0/3704163187 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fae3403e510 0x7fae340409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae40006fd0 tx=0x7fae40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.155+0000 7fae497fa700 1 -- 172.21.15.122:0/3704163187 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fae3c013070 con 0x7fae4c101fe0 2024-05-03T17:45:35.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:35.360+0000 7fae52343700 1 -- 172.21.15.122:0/3704163187 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) -- 0x7fae4c06aec0 con 0x7fae3403e510 2024-05-03T17:45:36.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:35 smithi122 ceph-mon[29744]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:45:37.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:36 smithi122 ceph-mon[29744]: Deploying cephadm binary to smithi149 2024-05-03T17:45:39.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:39.627+0000 7fae497fa700 1 -- 172.21.15.122:0/3704163187 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fae4c06aec0 con 0x7fae3403e510 2024-05-03T17:45:39.624 INFO:teuthology.orchestra.run.smithi122.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-05-03T17:45:39.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:39.630+0000 7fae52343700 1 -- 172.21.15.122:0/3704163187 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fae3403e510 msgr2=0x7fae340409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:39.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:39.630+0000 7fae52343700 1 --2- 172.21.15.122:0/3704163187 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fae3403e510 0x7fae340409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae40006fd0 tx=0x7fae40006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:39.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:39.630+0000 7fae52343700 1 -- 172.21.15.122:0/3704163187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4c101fe0 msgr2=0x7fae4c199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:39.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:39.630+0000 7fae52343700 1 --2- 172.21.15.122:0/3704163187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4c101fe0 0x7fae4c199720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae3c006010 tx=0x7fae3c005780 comp rx=0 tx=0).stop 2024-05-03T17:45:39.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:39.630+0000 7fae52343700 1 -- 172.21.15.122:0/3704163187 shutdown_connections 2024-05-03T17:45:39.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:39.630+0000 7fae52343700 1 --2- 172.21.15.122:0/3704163187 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fae3403e510 0x7fae340409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:39.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:39.630+0000 7fae52343700 1 --2- 172.21.15.122:0/3704163187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4c101fe0 0x7fae4c199720 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:39.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:39.630+0000 7fae52343700 1 -- 172.21.15.122:0/3704163187 >> 172.21.15.122:0/3704163187 conn(0x7fae4c078a70 msgr2=0x7fae4c1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:39.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:39.630+0000 7fae52343700 1 -- 172.21.15.122:0/3704163187 shutdown_connections 2024-05-03T17:45:39.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:39.630+0000 7fae52343700 1 -- 172.21.15.122:0/3704163187 wait complete. 2024-05-03T17:45:39.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:39 smithi122 ceph-mon[29744]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:45:40.203 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-03T17:45:40.434 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:45:40.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:40.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:40 smithi122 ceph-mon[29744]: Added host smithi149 2024-05-03T17:45:40.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:45:40.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:41.786 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:41 smithi122 ceph-mon[29744]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:45:41.786 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:42.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.024+0000 7fb718424700 1 -- 172.21.15.122:0/938044058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7101030f0 msgr2=0x7fb7101034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:42.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.024+0000 7fb718424700 1 --2- 172.21.15.122:0/938044058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7101030f0 0x7fb7101034d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009910 tx=0x7fb70c009c20 comp rx=0 tx=0).stop 2024-05-03T17:45:42.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.024+0000 7fb718424700 1 -- 172.21.15.122:0/938044058 shutdown_connections 2024-05-03T17:45:42.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.024+0000 7fb718424700 1 --2- 172.21.15.122:0/938044058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7101030f0 0x7fb7101034d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:42.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.024+0000 7fb718424700 1 -- 172.21.15.122:0/938044058 >> 172.21.15.122:0/938044058 conn(0x7fb7100fe9c0 msgr2=0x7fb710100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:42.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.024+0000 7fb718424700 1 -- 172.21.15.122:0/938044058 shutdown_connections 2024-05-03T17:45:42.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.025+0000 7fb718424700 1 -- 172.21.15.122:0/938044058 wait complete. 2024-05-03T17:45:42.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.025+0000 7fb718424700 1 Processor -- start 2024-05-03T17:45:42.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.025+0000 7fb718424700 1 -- start start 2024-05-03T17:45:42.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb718424700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7101030f0 0x7fb71019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:42.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb718424700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb71010d370 con 0x7fb7101030f0 2024-05-03T17:45:42.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb7161c0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7101030f0 0x7fb71019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:42.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb7161c0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7101030f0 0x7fb71019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38248/0 (socket says 172.21.15.122:38248) 2024-05-03T17:45:42.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb7161c0700 1 -- 172.21.15.122:0/1933164414 learned_addr learned my addr 172.21.15.122:0/1933164414 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:42.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb7161c0700 1 -- 172.21.15.122:0/1933164414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71019eb80 con 0x7fb7101030f0 2024-05-03T17:45:42.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb7161c0700 1 --2- 172.21.15.122:0/1933164414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7101030f0 0x7fb71019e640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0056d0 tx=0x7fb70c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:42.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb7077fe700 1 -- 172.21.15.122:0/1933164414 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb70c026070 con 0x7fb7101030f0 2024-05-03T17:45:42.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb718424700 1 -- 172.21.15.122:0/1933164414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71019ee70 con 0x7fb7101030f0 2024-05-03T17:45:42.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb7077fe700 1 -- 172.21.15.122:0/1933164414 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb70c00bba0 con 0x7fb7101030f0 2024-05-03T17:45:42.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb7077fe700 1 -- 172.21.15.122:0/1933164414 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb70c003920 con 0x7fb7101030f0 2024-05-03T17:45:42.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.026+0000 7fb718424700 1 -- 172.21.15.122:0/1933164414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7101988e0 con 0x7fb7101030f0 2024-05-03T17:45:42.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.027+0000 7fb7077fe700 1 -- 172.21.15.122:0/1933164414 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fb70c029070 con 0x7fb7101030f0 2024-05-03T17:45:42.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.027+0000 7fb718424700 1 -- 172.21.15.122:0/1933164414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7101045d0 con 0x7fb7101030f0 2024-05-03T17:45:42.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.027+0000 7fb7077fe700 1 --2- 172.21.15.122:0/1933164414 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb6fc03e510 0x7fb6fc0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:42.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.027+0000 7fb7077fe700 1 -- 172.21.15.122:0/1933164414 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb70c01e070 con 0x7fb7101030f0 2024-05-03T17:45:42.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.027+0000 7fb7159bf700 1 --2- 172.21.15.122:0/1933164414 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb6fc03e510 0x7fb6fc0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:42.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.028+0000 7fb7159bf700 1 --2- 172.21.15.122:0/1933164414 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb6fc03e510 0x7fb6fc0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb700006fd0 tx=0x7fb700006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:42.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.032+0000 7fb7077fe700 1 -- 172.21.15.122:0/1933164414 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb70c013070 con 0x7fb7101030f0 2024-05-03T17:45:42.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.242+0000 7fb718424700 1 -- 172.21.15.122:0/1933164414 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb710199cb0 con 0x7fb6fc03e510 2024-05-03T17:45:42.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.244+0000 7fb7077fe700 1 -- 172.21.15.122:0/1933164414 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fb710199cb0 con 0x7fb6fc03e510 2024-05-03T17:45:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:42.240 INFO:teuthology.orchestra.run.smithi122.stdout:[{"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-05-03T17:45:42.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.246+0000 7fb718424700 1 -- 172.21.15.122:0/1933164414 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb6fc03e510 msgr2=0x7fb6fc0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:42.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.246+0000 7fb718424700 1 --2- 172.21.15.122:0/1933164414 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb6fc03e510 0x7fb6fc0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb700006fd0 tx=0x7fb700006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:42.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.246+0000 7fb718424700 1 -- 172.21.15.122:0/1933164414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7101030f0 msgr2=0x7fb71019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:42.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.246+0000 7fb718424700 1 --2- 172.21.15.122:0/1933164414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7101030f0 0x7fb71019e640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0056d0 tx=0x7fb70c0057b0 comp rx=0 tx=0).stop 2024-05-03T17:45:42.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.246+0000 7fb718424700 1 -- 172.21.15.122:0/1933164414 shutdown_connections 2024-05-03T17:45:42.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.246+0000 7fb718424700 1 --2- 172.21.15.122:0/1933164414 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb6fc03e510 0x7fb6fc0409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:42.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.246+0000 7fb718424700 1 --2- 172.21.15.122:0/1933164414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7101030f0 0x7fb71019e640 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:42.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.246+0000 7fb718424700 1 -- 172.21.15.122:0/1933164414 >> 172.21.15.122:0/1933164414 conn(0x7fb7100fe9c0 msgr2=0x7fb710104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:42.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.246+0000 7fb718424700 1 -- 172.21.15.122:0/1933164414 shutdown_connections 2024-05-03T17:45:42.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:42.246+0000 7fb718424700 1 -- 172.21.15.122:0/1933164414 wait complete. 2024-05-03T17:45:42.776 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2024-05-03T17:45:42.776 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-03T17:45:42.776 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2024-05-03T17:45:42.794 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-03T17:45:42.795 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:45:42.852 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2024-05-03T17:45:42.852 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch host add smithi192 2024-05-03T17:45:43.083 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:45:43.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:43 smithi122 ceph-mon[29744]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:45:43.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:44.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.398+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/1230405419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d381030f0 msgr2=0x7f5d381034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:44.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.398+0000 7f5d3c8e4700 1 --2- 172.21.15.122:0/1230405419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d381030f0 0x7f5d381034d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009910 tx=0x7f5d2c009c20 comp rx=0 tx=0).stop 2024-05-03T17:45:44.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.398+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/1230405419 shutdown_connections 2024-05-03T17:45:44.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.398+0000 7f5d3c8e4700 1 --2- 172.21.15.122:0/1230405419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d381030f0 0x7f5d381034d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:44.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.398+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/1230405419 >> 172.21.15.122:0/1230405419 conn(0x7f5d380fe9c0 msgr2=0x7f5d38100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:44.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.398+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/1230405419 shutdown_connections 2024-05-03T17:45:44.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.398+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/1230405419 wait complete. 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.399+0000 7f5d3c8e4700 1 Processor -- start 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.399+0000 7f5d3c8e4700 1 -- start start 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.399+0000 7f5d3c8e4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d381030f0 0x7f5d380757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.399+0000 7f5d3c8e4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3810d370 con 0x7f5d381030f0 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.399+0000 7f5d3659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d381030f0 0x7f5d380757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.399+0000 7f5d3659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d381030f0 0x7f5d380757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38280/0 (socket says 172.21.15.122:38280) 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.399+0000 7f5d3659c700 1 -- 172.21.15.122:0/3216837386 learned_addr learned my addr 172.21.15.122:0/3216837386 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.400+0000 7f5d3659c700 1 -- 172.21.15.122:0/3216837386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d38077690 con 0x7f5d381030f0 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.400+0000 7f5d3659c700 1 --2- 172.21.15.122:0/3216837386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d381030f0 0x7f5d380757a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0056d0 tx=0x7f5d2c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.400+0000 7f5d277fe700 1 -- 172.21.15.122:0/3216837386 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d2c026070 con 0x7f5d381030f0 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.400+0000 7f5d277fe700 1 -- 172.21.15.122:0/3216837386 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d2c00bc00 con 0x7f5d381030f0 2024-05-03T17:45:44.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.400+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/3216837386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d38075d40 con 0x7f5d381030f0 2024-05-03T17:45:44.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.400+0000 7f5d277fe700 1 -- 172.21.15.122:0/3216837386 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d2c003920 con 0x7f5d381030f0 2024-05-03T17:45:44.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.400+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/3216837386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d38076200 con 0x7f5d381030f0 2024-05-03T17:45:44.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.401+0000 7f5d277fe700 1 -- 172.21.15.122:0/3216837386 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f5d2c029070 con 0x7f5d381030f0 2024-05-03T17:45:44.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.401+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/3216837386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d381045d0 con 0x7f5d381030f0 2024-05-03T17:45:44.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.401+0000 7f5d277fe700 1 --2- 172.21.15.122:0/3216837386 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5d2003e510 0x7f5d200409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:44.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.401+0000 7f5d277fe700 1 -- 172.21.15.122:0/3216837386 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5d2c01e070 con 0x7f5d381030f0 2024-05-03T17:45:44.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.401+0000 7f5d35d9b700 1 --2- 172.21.15.122:0/3216837386 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5d2003e510 0x7f5d200409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:44.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.401+0000 7f5d35d9b700 1 --2- 172.21.15.122:0/3216837386 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5d2003e510 0x7f5d200409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5d28006fd0 tx=0x7f5d28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:44.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.406+0000 7f5d277fe700 1 -- 172.21.15.122:0/3216837386 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5d2c013070 con 0x7f5d381030f0 2024-05-03T17:45:44.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:44 smithi122 ceph-mon[29744]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:45:44.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:44.616+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/3216837386 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}) -- 0x7f5d38076490 con 0x7f5d2003e510 2024-05-03T17:45:46.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:46 smithi122 ceph-mon[29744]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:45:46.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:46 smithi122 ceph-mon[29744]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:45:46.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:46 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:46.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:46 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:46.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:46 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:46.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:46 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:46.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:46 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:45:46.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:46 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:45:46.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:46 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:45:46.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:46 smithi122 ceph-mon[29744]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-03T17:45:47.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:47 smithi122 ceph-mon[29744]: Deploying cephadm binary to smithi192 2024-05-03T17:45:47.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:47 smithi122 ceph-mon[29744]: Updating smithi149:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:45:47.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:47 smithi122 ceph-mon[29744]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:45:47.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:47 smithi122 ceph-mon[29744]: Updating smithi149:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T17:45:48.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:48 smithi122 ceph-mon[29744]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:45:48.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:48.978+0000 7f5d277fe700 1 -- 172.21.15.122:0/3216837386 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5d38076490 con 0x7f5d2003e510 2024-05-03T17:45:48.975 INFO:teuthology.orchestra.run.smithi122.stdout:Added host 'smithi192' with addr '172.21.15.192' 2024-05-03T17:45:48.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:48.981+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/3216837386 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5d2003e510 msgr2=0x7f5d200409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:48.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:48.981+0000 7f5d3c8e4700 1 --2- 172.21.15.122:0/3216837386 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5d2003e510 0x7f5d200409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5d28006fd0 tx=0x7f5d28006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:48.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:48.981+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/3216837386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d381030f0 msgr2=0x7f5d380757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:48.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:48.981+0000 7f5d3c8e4700 1 --2- 172.21.15.122:0/3216837386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d381030f0 0x7f5d380757a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0056d0 tx=0x7f5d2c0057b0 comp rx=0 tx=0).stop 2024-05-03T17:45:48.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:48.981+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/3216837386 shutdown_connections 2024-05-03T17:45:48.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:48.981+0000 7f5d3c8e4700 1 --2- 172.21.15.122:0/3216837386 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5d2003e510 0x7f5d200409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:48.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:48.981+0000 7f5d3c8e4700 1 --2- 172.21.15.122:0/3216837386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d381030f0 0x7f5d380757a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:48.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:48.981+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/3216837386 >> 172.21.15.122:0/3216837386 conn(0x7f5d380fe9c0 msgr2=0x7f5d38104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:48.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:48.981+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/3216837386 shutdown_connections 2024-05-03T17:45:48.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:48.981+0000 7f5d3c8e4700 1 -- 172.21.15.122:0/3216837386 wait complete. 2024-05-03T17:45:49.696 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:49 smithi122 ceph-mon[29744]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:45:49.696 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:49.696 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:49.696 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:49.696 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:49.696 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:49 smithi122 ceph-mon[29744]: Added host smithi192 2024-05-03T17:45:49.696 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:45:49.724 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-03T17:45:49.958 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:45:50.593 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:50.593 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:51.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.283+0000 7f5fec796700 1 -- 172.21.15.122:0/4153230510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fe4102010 msgr2=0x7f5fe41023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:51.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.283+0000 7f5fec796700 1 --2- 172.21.15.122:0/4153230510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fe4102010 0x7f5fe41023f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009910 tx=0x7f5fd4009c20 comp rx=0 tx=0).stop 2024-05-03T17:45:51.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.283+0000 7f5fec796700 1 -- 172.21.15.122:0/4153230510 shutdown_connections 2024-05-03T17:45:51.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.283+0000 7f5fec796700 1 --2- 172.21.15.122:0/4153230510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fe4102010 0x7f5fe41023f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:51.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.283+0000 7f5fec796700 1 -- 172.21.15.122:0/4153230510 >> 172.21.15.122:0/4153230510 conn(0x7f5fe40fd8e0 msgr2=0x7f5fe40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:51.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.283+0000 7f5fec796700 1 -- 172.21.15.122:0/4153230510 shutdown_connections 2024-05-03T17:45:51.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.283+0000 7f5fec796700 1 -- 172.21.15.122:0/4153230510 wait complete. 2024-05-03T17:45:51.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.284+0000 7f5fec796700 1 Processor -- start 2024-05-03T17:45:51.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.284+0000 7f5fec796700 1 -- start start 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.284+0000 7f5fec796700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fe4102010 0x7f5fe419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.284+0000 7f5fec796700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe410c290 con 0x7f5fe4102010 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.284+0000 7f5fea532700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fe4102010 0x7f5fe419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.284+0000 7f5fea532700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fe4102010 0x7f5fe419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44952/0 (socket says 172.21.15.122:44952) 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.284+0000 7f5fea532700 1 -- 172.21.15.122:0/2989246139 learned_addr learned my addr 172.21.15.122:0/2989246139 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.284+0000 7f5fea532700 1 -- 172.21.15.122:0/2989246139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe419eb40 con 0x7f5fe4102010 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.284+0000 7f5fea532700 1 --2- 172.21.15.122:0/2989246139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fe4102010 0x7f5fe419e600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4006010 tx=0x7f5fd4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.285+0000 7f5fdb7fe700 1 -- 172.21.15.122:0/2989246139 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fd4026070 con 0x7f5fe4102010 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.285+0000 7f5fec796700 1 -- 172.21.15.122:0/2989246139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe419ee30 con 0x7f5fe4102010 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.285+0000 7f5fdb7fe700 1 -- 172.21.15.122:0/2989246139 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fd400bba0 con 0x7f5fe4102010 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.285+0000 7f5fdb7fe700 1 -- 172.21.15.122:0/2989246139 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fd4003910 con 0x7f5fe4102010 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.285+0000 7f5fec796700 1 -- 172.21.15.122:0/2989246139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe41988a0 con 0x7f5fe4102010 2024-05-03T17:45:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.286+0000 7f5fec796700 1 -- 172.21.15.122:0/2989246139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe41034f0 con 0x7f5fe4102010 2024-05-03T17:45:51.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.286+0000 7f5fdb7fe700 1 -- 172.21.15.122:0/2989246139 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f5fd4029070 con 0x7f5fe4102010 2024-05-03T17:45:51.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.286+0000 7f5fdb7fe700 1 --2- 172.21.15.122:0/2989246139 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5fd003e510 0x7f5fd00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:51.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.286+0000 7f5fdb7fe700 1 -- 172.21.15.122:0/2989246139 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5fd401e070 con 0x7f5fe4102010 2024-05-03T17:45:51.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.286+0000 7f5fe9d31700 1 --2- 172.21.15.122:0/2989246139 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5fd003e510 0x7f5fd00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:51.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.287+0000 7f5fe9d31700 1 --2- 172.21.15.122:0/2989246139 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5fd003e510 0x7f5fd00409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0006fd0 tx=0x7f5fe0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:51.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.291+0000 7f5fdb7fe700 1 -- 172.21.15.122:0/2989246139 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5fd4013070 con 0x7f5fe4102010 2024-05-03T17:45:51.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.500+0000 7f5fec796700 1 -- 172.21.15.122:0/2989246139 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5fe4199c30 con 0x7f5fd003e510 2024-05-03T17:45:51.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.501+0000 7f5fdb7fe700 1 -- 172.21.15.122:0/2989246139 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f5fe4199c30 con 0x7f5fd003e510 2024-05-03T17:45:51.497 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:45:51.497 INFO:teuthology.orchestra.run.smithi122.stdout:[{"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2024-05-03T17:45:51.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.504+0000 7f5fec796700 1 -- 172.21.15.122:0/2989246139 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5fd003e510 msgr2=0x7f5fd00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:51.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.504+0000 7f5fec796700 1 --2- 172.21.15.122:0/2989246139 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5fd003e510 0x7f5fd00409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0006fd0 tx=0x7f5fe0006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:51.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.504+0000 7f5fec796700 1 -- 172.21.15.122:0/2989246139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fe4102010 msgr2=0x7f5fe419e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:51.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.504+0000 7f5fec796700 1 --2- 172.21.15.122:0/2989246139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fe4102010 0x7f5fe419e600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4006010 tx=0x7f5fd4005780 comp rx=0 tx=0).stop 2024-05-03T17:45:51.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.504+0000 7f5fec796700 1 -- 172.21.15.122:0/2989246139 shutdown_connections 2024-05-03T17:45:51.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.504+0000 7f5fec796700 1 --2- 172.21.15.122:0/2989246139 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5fd003e510 0x7f5fd00409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:51.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.504+0000 7f5fec796700 1 --2- 172.21.15.122:0/2989246139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fe4102010 0x7f5fe419e600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:51.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.504+0000 7f5fec796700 1 -- 172.21.15.122:0/2989246139 >> 172.21.15.122:0/2989246139 conn(0x7f5fe40fd8e0 msgr2=0x7f5fe406b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:51.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.504+0000 7f5fec796700 1 -- 172.21.15.122:0/2989246139 shutdown_connections 2024-05-03T17:45:51.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:51.504+0000 7f5fec796700 1 -- 172.21.15.122:0/2989246139 wait complete. 2024-05-03T17:45:52.067 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:52 smithi122 ceph-mon[29744]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:45:52.067 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:52.068 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-03T17:45:52.069 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-03T17:45:52.298 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:45:53.297 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:53 smithi122 ceph-mon[29744]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:45:53.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.782+0000 7f6f93ba4700 1 -- 172.21.15.122:0/2111531889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f8c1030f0 msgr2=0x7f6f8c1034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:53.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.782+0000 7f6f93ba4700 1 --2- 172.21.15.122:0/2111531889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f8c1030f0 0x7f6f8c1034d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009910 tx=0x7f6f88009c20 comp rx=0 tx=0).stop 2024-05-03T17:45:53.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.782+0000 7f6f93ba4700 1 -- 172.21.15.122:0/2111531889 shutdown_connections 2024-05-03T17:45:53.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.782+0000 7f6f93ba4700 1 --2- 172.21.15.122:0/2111531889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f8c1030f0 0x7f6f8c1034d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:53.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.782+0000 7f6f93ba4700 1 -- 172.21.15.122:0/2111531889 >> 172.21.15.122:0/2111531889 conn(0x7f6f8c0fe9c0 msgr2=0x7f6f8c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.783+0000 7f6f93ba4700 1 -- 172.21.15.122:0/2111531889 shutdown_connections 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.783+0000 7f6f93ba4700 1 -- 172.21.15.122:0/2111531889 wait complete. 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.783+0000 7f6f93ba4700 1 Processor -- start 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.783+0000 7f6f93ba4700 1 -- start start 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.783+0000 7f6f93ba4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f8c1030f0 0x7f6f8c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.783+0000 7f6f93ba4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c10d370 con 0x7f6f8c1030f0 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.783+0000 7f6f91940700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f8c1030f0 0x7f6f8c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.783+0000 7f6f91940700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f8c1030f0 0x7f6f8c19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44968/0 (socket says 172.21.15.122:44968) 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.783+0000 7f6f91940700 1 -- 172.21.15.122:0/3772868751 learned_addr learned my addr 172.21.15.122:0/3772868751 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.784+0000 7f6f91940700 1 -- 172.21.15.122:0/3772868751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c19eb70 con 0x7f6f8c1030f0 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.784+0000 7f6f91940700 1 --2- 172.21.15.122:0/3772868751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f8c1030f0 0x7f6f8c19e630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f880056d0 tx=0x7f6f880057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.784+0000 7f6f82ffd700 1 -- 172.21.15.122:0/3772868751 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f88026070 con 0x7f6f8c1030f0 2024-05-03T17:45:53.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.784+0000 7f6f93ba4700 1 -- 172.21.15.122:0/3772868751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c19ee60 con 0x7f6f8c1030f0 2024-05-03T17:45:53.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.784+0000 7f6f82ffd700 1 -- 172.21.15.122:0/3772868751 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f8800bba0 con 0x7f6f8c1030f0 2024-05-03T17:45:53.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.784+0000 7f6f82ffd700 1 -- 172.21.15.122:0/3772868751 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f88003920 con 0x7f6f8c1030f0 2024-05-03T17:45:53.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.784+0000 7f6f93ba4700 1 -- 172.21.15.122:0/3772868751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c1988d0 con 0x7f6f8c1030f0 2024-05-03T17:45:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.785+0000 7f6f82ffd700 1 -- 172.21.15.122:0/3772868751 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f6f88029070 con 0x7f6f8c1030f0 2024-05-03T17:45:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.785+0000 7f6f93ba4700 1 -- 172.21.15.122:0/3772868751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f8c1045d0 con 0x7f6f8c1030f0 2024-05-03T17:45:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.785+0000 7f6f82ffd700 1 --2- 172.21.15.122:0/3772868751 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f6f7803e510 0x7f6f780409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.785+0000 7f6f82ffd700 1 -- 172.21.15.122:0/3772868751 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f8801e070 con 0x7f6f8c1030f0 2024-05-03T17:45:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.785+0000 7f6f9113f700 1 --2- 172.21.15.122:0/3772868751 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f6f7803e510 0x7f6f780409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.786+0000 7f6f9113f700 1 --2- 172.21.15.122:0/3772868751 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f6f7803e510 0x7f6f780409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c006fd0 tx=0x7f6f7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.790+0000 7f6f82ffd700 1 -- 172.21.15.122:0/3772868751 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6f88013070 con 0x7f6f8c1030f0 2024-05-03T17:45:53.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:53.999+0000 7f6f93ba4700 1 -- 172.21.15.122:0/3772868751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f6f8c199c30 con 0x7f6f8c1030f0 2024-05-03T17:45:54.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:54.038+0000 7f6f82ffd700 1 -- 172.21.15.122:0/3772868751 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f6f8800d340 con 0x7f6f8c1030f0 2024-05-03T17:45:54.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:54.040+0000 7f6f93ba4700 1 -- 172.21.15.122:0/3772868751 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f6f7803e510 msgr2=0x7f6f780409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:54.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:54.040+0000 7f6f93ba4700 1 --2- 172.21.15.122:0/3772868751 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f6f7803e510 0x7f6f780409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c006fd0 tx=0x7f6f7c006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:54.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:54.040+0000 7f6f93ba4700 1 -- 172.21.15.122:0/3772868751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f8c1030f0 msgr2=0x7f6f8c19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:54.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:54.040+0000 7f6f93ba4700 1 --2- 172.21.15.122:0/3772868751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f8c1030f0 0x7f6f8c19e630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f880056d0 tx=0x7f6f880057b0 comp rx=0 tx=0).stop 2024-05-03T17:45:54.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:54.041+0000 7f6f93ba4700 1 -- 172.21.15.122:0/3772868751 shutdown_connections 2024-05-03T17:45:54.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:54.041+0000 7f6f93ba4700 1 --2- 172.21.15.122:0/3772868751 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f6f7803e510 0x7f6f780409d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:54.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:54.041+0000 7f6f93ba4700 1 --2- 172.21.15.122:0/3772868751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f8c1030f0 0x7f6f8c19e630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:54.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:54.041+0000 7f6f93ba4700 1 -- 172.21.15.122:0/3772868751 >> 172.21.15.122:0/3772868751 conn(0x7f6f8c0fe9c0 msgr2=0x7f6f8c104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:54.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:54.041+0000 7f6f93ba4700 1 -- 172.21.15.122:0/3772868751 shutdown_connections 2024-05-03T17:45:54.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:45:54.041+0000 7f6f93ba4700 1 -- 172.21.15.122:0/3772868751 wait complete. 2024-05-03T17:45:54.037 INFO:teuthology.orchestra.run.smithi122.stderr:adjusted tunables profile to default 2024-05-03T17:45:54.428 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:54 smithi122 ceph-mon[29744]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:45:54.428 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:54 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3772868751' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T17:45:54.665 INFO:tasks.cephadm:Adding mon.a on smithi122 2024-05-03T17:45:54.665 INFO:tasks.cephadm:Adding mon.b on smithi149 2024-05-03T17:45:54.665 INFO:tasks.cephadm:Adding mon.c on smithi192 2024-05-03T17:45:54.665 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi122:172.21.15.122=a;smithi149:172.21.15.149=b;smithi192:172.21.15.192=c' 2024-05-03T17:45:54.966 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-03T17:45:55.021 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-03T17:45:55.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:55 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3772868751' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T17:45:55.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:55 smithi122 ceph-mon[29744]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T17:45:55.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:55.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:55.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:55.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:55.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:45:55.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:45:55.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:45:56.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:56 smithi122 ceph-mon[29744]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:45:56.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:56 smithi122 ceph-mon[29744]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-03T17:45:56.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:56 smithi122 ceph-mon[29744]: Updating smithi192:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:45:56.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:56 smithi122 ceph-mon[29744]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:45:56.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:56 smithi122 ceph-mon[29744]: Updating smithi192:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T17:45:56.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:56.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:56.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:56.605 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.608+0000 7f602be6a700 1 -- 172.21.15.192:0/3898828655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6024101e60 msgr2=0x7f6024102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:56.605 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.608+0000 7f602be6a700 1 --2- 172.21.15.192:0/3898828655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6024101e60 0x7f6024102240 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6020009910 tx=0x7f6020009c20 comp rx=0 tx=0).stop 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.608+0000 7f602be6a700 1 -- 172.21.15.192:0/3898828655 shutdown_connections 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.608+0000 7f602be6a700 1 --2- 172.21.15.192:0/3898828655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6024101e60 0x7f6024102240 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.608+0000 7f602be6a700 1 -- 172.21.15.192:0/3898828655 >> 172.21.15.192:0/3898828655 conn(0x7f60240fd8e0 msgr2=0x7f60240ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.608+0000 7f602be6a700 1 -- 172.21.15.192:0/3898828655 shutdown_connections 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.608+0000 7f602be6a700 1 -- 172.21.15.192:0/3898828655 wait complete. 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.609+0000 7f602be6a700 1 Processor -- start 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.609+0000 7f602be6a700 1 -- start start 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.609+0000 7f602be6a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6024101e60 0x7f602419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.609+0000 7f602be6a700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6024113a10 con 0x7f6024101e60 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.609+0000 7f6029c06700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6024101e60 0x7f602419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.609+0000 7f6029c06700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6024101e60 0x7f602419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.192:56938/0 (socket says 172.21.15.192:56938) 2024-05-03T17:45:56.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.609+0000 7f6029c06700 1 -- 172.21.15.192:0/1670070293 learned_addr learned my addr 172.21.15.192:0/1670070293 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-03T17:45:56.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.610+0000 7f6029c06700 1 -- 172.21.15.192:0/1670070293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602419f490 con 0x7f6024101e60 2024-05-03T17:45:56.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.610+0000 7f6029c06700 1 --2- 172.21.15.192:0/1670070293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6024101e60 0x7f602419ef50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f60200056d0 tx=0x7f60200057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:56.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.610+0000 7f601affd700 1 -- 172.21.15.192:0/1670070293 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6020026070 con 0x7f6024101e60 2024-05-03T17:45:56.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.610+0000 7f601affd700 1 -- 172.21.15.192:0/1670070293 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6020004370 con 0x7f6024101e60 2024-05-03T17:45:56.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.610+0000 7f601affd700 1 -- 172.21.15.192:0/1670070293 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60200039b0 con 0x7f6024101e60 2024-05-03T17:45:56.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.610+0000 7f602be6a700 1 -- 172.21.15.192:0/1670070293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602419f720 con 0x7f6024101e60 2024-05-03T17:45:56.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.610+0000 7f602be6a700 1 -- 172.21.15.192:0/1670070293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f602419fb80 con 0x7f6024101e60 2024-05-03T17:45:56.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.611+0000 7f601affd700 1 -- 172.21.15.192:0/1670070293 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f6020029070 con 0x7f6024101e60 2024-05-03T17:45:56.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.611+0000 7f602be6a700 1 -- 172.21.15.192:0/1670070293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f602406ba40 con 0x7f6024101e60 2024-05-03T17:45:56.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.611+0000 7f601affd700 1 --2- 172.21.15.192:0/1670070293 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f601003e510 0x7f60100409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:45:56.609 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.611+0000 7f601affd700 1 -- 172.21.15.192:0/1670070293 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f602001e070 con 0x7f6024101e60 2024-05-03T17:45:56.609 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.611+0000 7f6029405700 1 --2- 172.21.15.192:0/1670070293 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f601003e510 0x7f60100409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:45:56.609 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.612+0000 7f6029405700 1 --2- 172.21.15.192:0/1670070293 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f601003e510 0x7f60100409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6014006fd0 tx=0x7f6014006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:45:56.612 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.617+0000 7f601affd700 1 -- 172.21.15.192:0/1670070293 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6020013070 con 0x7f6024101e60 2024-05-03T17:45:56.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.831+0000 7f602be6a700 1 -- 172.21.15.192:0/1670070293 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi149:172.21.15.149=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}) -- 0x7f60240ff9f0 con 0x7f601003e510 2024-05-03T17:45:56.833 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.838+0000 7f601affd700 1 -- 172.21.15.192:0/1670070293 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f60240ff9f0 con 0x7f601003e510 2024-05-03T17:45:56.834 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2024-05-03T17:45:56.835 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.839+0000 7f602be6a700 1 -- 172.21.15.192:0/1670070293 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f601003e510 msgr2=0x7f60100409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:56.835 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.839+0000 7f602be6a700 1 --2- 172.21.15.192:0/1670070293 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f601003e510 0x7f60100409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6014006fd0 tx=0x7f6014006e40 comp rx=0 tx=0).stop 2024-05-03T17:45:56.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.839+0000 7f602be6a700 1 -- 172.21.15.192:0/1670070293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6024101e60 msgr2=0x7f602419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:45:56.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.839+0000 7f602be6a700 1 --2- 172.21.15.192:0/1670070293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6024101e60 0x7f602419ef50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f60200056d0 tx=0x7f60200057b0 comp rx=0 tx=0).stop 2024-05-03T17:45:56.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.840+0000 7f602be6a700 1 -- 172.21.15.192:0/1670070293 shutdown_connections 2024-05-03T17:45:56.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.840+0000 7f602be6a700 1 --2- 172.21.15.192:0/1670070293 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f601003e510 0x7f60100409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:56.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.840+0000 7f602be6a700 1 --2- 172.21.15.192:0/1670070293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6024101e60 0x7f602419ef50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:45:56.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.840+0000 7f602be6a700 1 -- 172.21.15.192:0/1670070293 >> 172.21.15.192:0/1670070293 conn(0x7f60240fd8e0 msgr2=0x7f60240ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:45:56.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.840+0000 7f602be6a700 1 -- 172.21.15.192:0/1670070293 shutdown_connections 2024-05-03T17:45:56.836 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:45:56.840+0000 7f602be6a700 1 -- 172.21.15.192:0/1670070293 wait complete. 2024-05-03T17:45:57.435 DEBUG:teuthology.orchestra.run.smithi149:mon.b> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.b.service 2024-05-03T17:45:57.437 DEBUG:teuthology.orchestra.run.smithi192:mon.c> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.c.service 2024-05-03T17:45:57.440 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T17:45:57.440 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T17:45:58.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:57 smithi122 ceph-mon[29744]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:45:58.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:57 smithi122 ceph-mon[29744]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi149:172.21.15.149=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:45:58.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:57 smithi122 ceph-mon[29744]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi149:172.21.15.149=b;smithi192:172.21.15.192=c;count:3 2024-05-03T17:45:58.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:58.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:45:58.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:45:58.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:45:58.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:45:58.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:45:58.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:45:58.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:57 smithi122 ceph-mon[29744]: Deploying daemon mon.c on smithi192 2024-05-03T17:45:59.104 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:00.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:45:59 smithi122 ceph-mon[29744]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:01.044 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:00 smithi192 systemd[1]: Starting Ceph mon.c for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:46:01.416 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.420+0000 7f386098d700 1 -- 172.21.15.192:0/3079914520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f385c103120 msgr2=0x7f385c103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:01.416 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.420+0000 7f386098d700 1 --2- 172.21.15.192:0/3079914520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f385c103120 0x7f385c103500 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f384c009910 tx=0x7f384c009c20 comp rx=0 tx=0).stop 2024-05-03T17:46:01.417 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.420+0000 7f386098d700 1 -- 172.21.15.192:0/3079914520 shutdown_connections 2024-05-03T17:46:01.417 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.420+0000 7f386098d700 1 --2- 172.21.15.192:0/3079914520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f385c103120 0x7f385c103500 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:01.417 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.420+0000 7f386098d700 1 -- 172.21.15.192:0/3079914520 >> 172.21.15.192:0/3079914520 conn(0x7f385c0fe9d0 msgr2=0x7f385c100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:01.417 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.420+0000 7f386098d700 1 -- 172.21.15.192:0/3079914520 shutdown_connections 2024-05-03T17:46:01.417 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.420+0000 7f386098d700 1 -- 172.21.15.192:0/3079914520 wait complete. 2024-05-03T17:46:01.417 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.421+0000 7f386098d700 1 Processor -- start 2024-05-03T17:46:01.417 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.421+0000 7f386098d700 1 -- start start 2024-05-03T17:46:01.417 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.421+0000 7f386098d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f385c103120 0x7f385c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:01.417 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.421+0000 7f386098d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f385c10d3a0 con 0x7f385c103120 2024-05-03T17:46:01.418 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.421+0000 7f385a59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f385c103120 0x7f385c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:01.418 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.421+0000 7f385a59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f385c103120 0x7f385c1a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.192:56954/0 (socket says 172.21.15.192:56954) 2024-05-03T17:46:01.418 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.421+0000 7f385a59c700 1 -- 172.21.15.192:0/4189622796 learned_addr learned my addr 172.21.15.192:0/4189622796 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-03T17:46:01.418 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.421+0000 7f385a59c700 1 -- 172.21.15.192:0/4189622796 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c1a0cc0 con 0x7f385c103120 2024-05-03T17:46:01.419 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.422+0000 7f385a59c700 1 --2- 172.21.15.192:0/4189622796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f385c103120 0x7f385c1a0780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f384c0056d0 tx=0x7f384c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:01.419 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.422+0000 7f384b7fe700 1 -- 172.21.15.192:0/4189622796 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f384c026070 con 0x7f385c103120 2024-05-03T17:46:01.419 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.422+0000 7f384b7fe700 1 -- 172.21.15.192:0/4189622796 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f384c00bba0 con 0x7f385c103120 2024-05-03T17:46:01.419 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.422+0000 7f386098d700 1 -- 172.21.15.192:0/4189622796 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385c1a0fb0 con 0x7f385c103120 2024-05-03T17:46:01.419 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.422+0000 7f384b7fe700 1 -- 172.21.15.192:0/4189622796 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f384c003920 con 0x7f385c103120 2024-05-03T17:46:01.419 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.422+0000 7f386098d700 1 -- 172.21.15.192:0/4189622796 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385c19a940 con 0x7f385c103120 2024-05-03T17:46:01.420 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.423+0000 7f386098d700 1 -- 172.21.15.192:0/4189622796 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385c104600 con 0x7f385c103120 2024-05-03T17:46:01.420 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.423+0000 7f384b7fe700 1 -- 172.21.15.192:0/4189622796 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f384c029070 con 0x7f385c103120 2024-05-03T17:46:01.420 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.423+0000 7f384b7fe700 1 --2- 172.21.15.192:0/4189622796 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f384403e510 0x7f38440409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:01.420 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.423+0000 7f384b7fe700 1 -- 172.21.15.192:0/4189622796 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f384c01e070 con 0x7f385c103120 2024-05-03T17:46:01.420 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.423+0000 7f3859d9b700 1 --2- 172.21.15.192:0/4189622796 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f384403e510 0x7f38440409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:01.420 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.424+0000 7f3859d9b700 1 --2- 172.21.15.192:0/4189622796 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f384403e510 0x7f38440409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3850006fd0 tx=0x7f3850006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:01.424 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.429+0000 7f384b7fe700 1 -- 172.21.15.192:0/4189622796 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f384c013070 con 0x7f385c103120 2024-05-03T17:46:01.591 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 podman[33712]: 2024-05-03 17:46:01.320663021 +0000 UTC m=+0.072812368 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:46:01.591 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 podman[33712]: 2024-05-03 17:46:01.499186761 +0000 UTC m=+0.251336104 container create cac55d07538833bcddc68a21dbcd5d36bcf32d849b27edd32e54362db352ab45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T17:46:01.721 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.725+0000 7f386098d700 1 -- 172.21.15.192:0/4189622796 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f385c19bf10 con 0x7f385c103120 2024-05-03T17:46:01.722 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.726+0000 7f384b7fe700 1 -- 172.21.15.192:0/4189622796 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f384c030090 con 0x7f385c103120 2024-05-03T17:46:01.722 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:46:01.722 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","modified":"2024-05-03T17:44:10.382958Z","created":"2024-05-03T17:44:10.382958Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-03T17:46:01.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.730+0000 7f386098d700 1 -- 172.21.15.192:0/4189622796 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f384403e510 msgr2=0x7f38440409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:01.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.730+0000 7f386098d700 1 --2- 172.21.15.192:0/4189622796 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f384403e510 0x7f38440409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3850006fd0 tx=0x7f3850006e40 comp rx=0 tx=0).stop 2024-05-03T17:46:01.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.730+0000 7f386098d700 1 -- 172.21.15.192:0/4189622796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f385c103120 msgr2=0x7f385c1a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:01.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.730+0000 7f386098d700 1 --2- 172.21.15.192:0/4189622796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f385c103120 0x7f385c1a0780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f384c0056d0 tx=0x7f384c0057b0 comp rx=0 tx=0).stop 2024-05-03T17:46:01.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.730+0000 7f386098d700 1 -- 172.21.15.192:0/4189622796 shutdown_connections 2024-05-03T17:46:01.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.730+0000 7f386098d700 1 --2- 172.21.15.192:0/4189622796 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f384403e510 0x7f38440409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:01.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.730+0000 7f386098d700 1 --2- 172.21.15.192:0/4189622796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f385c103120 0x7f385c1a0780 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:01.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.730+0000 7f386098d700 1 -- 172.21.15.192:0/4189622796 >> 172.21.15.192:0/4189622796 conn(0x7f385c0fe9d0 msgr2=0x7f385c0756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:01.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.730+0000 7f386098d700 1 -- 172.21.15.192:0/4189622796 shutdown_connections 2024-05-03T17:46:01.726 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:01.730+0000 7f386098d700 1 -- 172.21.15.192:0/4189622796 wait complete. 2024-05-03T17:46:01.727 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 podman[33712]: 2024-05-03 17:46:01.693275593 +0000 UTC m=+0.445424947 container init cac55d07538833bcddc68a21dbcd5d36bcf32d849b27edd32e54362db352ab45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 podman[33712]: 2024-05-03 17:46:01.69681177 +0000 UTC m=+0.448961115 container start cac55d07538833bcddc68a21dbcd5d36bcf32d849b27edd32e54362db352ab45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: set uid:gid to 167:167 (ceph:ceph) 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev), process ceph-mon, pid 2 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: pidfile_write: ignore empty --pid-file 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: load: jerasure load: lrc 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: RocksDB version: 7.9.2 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Git sha 0 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Compile date 2024-05-01 21:12:03 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: DB SUMMARY 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: DB Session ID: W3DT9TVPPXYKO848JXSI 2024-05-03T17:46:02.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: CURRENT file: CURRENT 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: IDENTITY file: IDENTITY 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.error_if_exists: 0 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.create_if_missing: 0 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.paranoid_checks: 1 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.env: 0x562d02d9ecc0 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.fs: PosixFileSystem 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.info_log: 0x562d03e1bda0 2024-05-03T17:46:02.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_file_opening_threads: 16 2024-05-03T17:46:02.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.statistics: (nil) 2024-05-03T17:46:02.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.use_fsync: 0 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_log_file_size: 0 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.keep_log_file_num: 1000 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.recycle_log_file_num: 0 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.allow_fallocate: 1 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.allow_mmap_reads: 0 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.allow_mmap_writes: 0 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.use_direct_reads: 0 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.create_missing_column_families: 0 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.db_log_dir: 2024-05-03T17:46:02.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.wal_dir: 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.advise_random_on_open: 1 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.db_write_buffer_size: 0 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.write_buffer_manager: 0x562d03ded0e0 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-03T17:46:02.088 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.rate_limiter: (nil) 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.wal_recovery_mode: 2 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.enable_thread_tracking: 0 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.enable_pipelined_write: 0 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.unordered_write: 0 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.row_cache: None 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.wal_filter: None 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.allow_ingest_behind: 0 2024-05-03T17:46:02.089 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.two_write_queues: 0 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.manual_wal_flush: 0 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.wal_compression: 0 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.atomic_flush: 0 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.log_readahead_size: 0 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.best_efforts_recovery: 0 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.allow_data_in_errors: 0 2024-05-03T17:46:02.090 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.db_host_id: __hostname__ 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_background_jobs: 2 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_background_compactions: -1 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_subcompactions: 1 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_total_wal_size: 0 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_open_files: -1 2024-05-03T17:46:02.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bytes_per_sync: 0 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_readahead_size: 0 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_background_flushes: -1 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Compression algorithms supported: 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: kZSTD supported: 0 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: kXpressCompression supported: 0 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: kLZ4HCCompression supported: 1 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: kZlibCompression supported: 1 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: kSnappyCompression supported: 1 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: kLZ4Compression supported: 1 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: kBZip2Compression supported: 0 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-03T17:46:02.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.merge_operator: 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_filter: None 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_filter_factory: None 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.sst_partitioner_factory: None 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562d03e1bf60) 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks: 1 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-03T17:46:02.093 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_top_level_index_and_filter: 1 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_type: 0 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_index_type: 0 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_shortening: 1 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: checksum: 4 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: no_block_cache: 0 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache: 0x562d03e1f1f0 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_name: BinnedLRUCache 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_options: 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: capacity : 536870912 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_shard_bits : 4 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: strict_capacity_limit : 0 2024-05-03T17:46:02.094 INFO:journalctl@ceph.mon.c.smithi192.stdout: high_pri_pool_ratio: 0.000 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_compressed: (nil) 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: persistent_cache: (nil) 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size: 4096 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size_deviation: 10 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_restart_interval: 16 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_block_restart_interval: 1 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: metadata_block_size: 4096 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: partition_filters: 0 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: use_delta_encoding: 1 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: filter_policy: bloomfilter 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: whole_key_filtering: 1 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: verify_compression: 0 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: read_amp_bytes_per_bit: 0 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: format_version: 5 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_index_compression: 1 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_align: 0 2024-05-03T17:46:02.095 INFO:journalctl@ceph.mon.c.smithi192.stdout: max_auto_readahead_size: 262144 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout: prepopulate_block_cache: 0 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout: initial_auto_readahead_size: 8192 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_file_reads_for_auto_readahead: 2 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.write_buffer_size: 33554432 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_write_buffer_number: 2 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compression: NoCompression 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bottommost_compression: Disabled 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.prefix_extractor: nullptr 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.num_levels: 7 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-03T17:46:02.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compression_opts.level: 32767 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compression_opts.strategy: 0 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compression_opts.enabled: false 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-03T17:46:02.097 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.target_file_size_base: 67108864 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-03T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.arena_block_size: 1048576 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.disable_auto_compactions: 0 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-03T17:46:02.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.inplace_update_support: 0 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.bloom_locality: 0 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.max_successive_merges: 0 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.paranoid_file_checks: 0 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.force_consistency_checks: 1 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.report_bg_io_stats: 0 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.ttl: 2592000 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-03T17:46:02.100 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.enable_blob_files: false 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.min_blob_size: 0 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.blob_file_size: 268435456 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.blob_file_starting_level: 0 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 80f1a7a1-bb7f-44ae-9208-f1c33c75e2fd 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714758361726933, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-03T17:46:02.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714758361727372, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714758361, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "80f1a7a1-bb7f-44ae-9208-f1c33c75e2fd", "db_session_id": "W3DT9TVPPXYKO848JXSI", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714758361727462, "job": 1, "event": "recovery_finished"} 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562d03f3c000 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: DB pointer 0x562d03f28000 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** DB Stats ** 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T17:46:02.102 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-03T17:46:02.103 INFO:journalctl@ceph.mon.c.smithi192.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache BinnedLRUCache@0x562d03e1f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-03T17:46:02.104 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: using public_addr v2:172.21.15.192:0/0 -> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: starting mon.c rank -1 at public addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] at bind addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(???) e0 preinit fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).mds e1 new map 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout: e1 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout: legacy client fscid: -1 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout: No filesystems configured 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-03T17:46:02.105 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.106 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Deploying cephadm binary to smithi192 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Updating smithi149:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Updating smithi149:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Added host smithi192 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3772868751' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3772868751' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Updating smithi192:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Updating smithi192:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi149:172.21.15.149=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi149:172.21.15.149=b;smithi192:172.21.15.192=c;count:3 2024-05-03T17:46:02.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.110 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:02.110 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:02.110 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:02.110 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:02.110 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:02.110 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:02.110 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: Deploying daemon mon.c on smithi192 2024-05-03T17:46:02.110 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:02.110 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 ceph-mon[33753]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-03T17:46:02.110 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 bash[33712]: cac55d07538833bcddc68a21dbcd5d36bcf32d849b27edd32e54362db352ab45 2024-05-03T17:46:02.110 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:01 smithi192 systemd[1]: Started Ceph mon.c for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:46:03.470 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T17:46:03.470 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T17:46:03.714 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:46:07.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:46:07.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: mon.a calling monitor election 2024-05-03T17:46:07.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:07.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: mon.c calling monitor election 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: monmap epoch 2 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: last_changed 2024-05-03T17:46:01.751424+0000 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: created 2024-05-03T17:44:10.382958+0000 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: min_mon_release 19 (squid) 2024-05-03T17:46:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: election_strategy: 1 2024-05-03T17:46:07.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-05-03T17:46:07.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-03T17:46:07.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: fsmap 2024-05-03T17:46:07.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T17:46:07.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: mgrmap e13: x(active, since 48s) 2024-05-03T17:46:07.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: overall HEALTH_OK 2024-05-03T17:46:07.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:07.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:07.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:07.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:07.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:07.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:46:07.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: mon.a calling monitor election 2024-05-03T17:46:07.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:07.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: mon.c calling monitor election 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: monmap epoch 2 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: last_changed 2024-05-03T17:46:01.751424+0000 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: created 2024-05-03T17:44:10.382958+0000 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: min_mon_release 19 (squid) 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: election_strategy: 1 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-03T17:46:07.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: fsmap 2024-05-03T17:46:07.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T17:46:07.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: mgrmap e13: x(active, since 48s) 2024-05-03T17:46:07.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: overall HEALTH_OK 2024-05-03T17:46:07.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:07.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:07.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:07.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:07.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:08.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:07 smithi192 ceph-mon[33753]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:08.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:07 smithi192 ceph-mon[33753]: Deploying daemon mon.b on smithi149 2024-05-03T17:46:08.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:07 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:08.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.207+0000 7f3707bab700 1 -- 172.21.15.192:0/3353754222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec0035c0 msgr2=0x7f36ec005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:08.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.207+0000 7f3707bab700 1 --2- 172.21.15.192:0/3353754222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec0035c0 0x7f36ec005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3700101dc0 tx=0x7f36fc00feb0 comp rx=0 tx=0).stop 2024-05-03T17:46:08.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.207+0000 7f3707bab700 1 -- 172.21.15.192:0/3353754222 shutdown_connections 2024-05-03T17:46:08.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.207+0000 7f3707bab700 1 --2- 172.21.15.192:0/3353754222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec0035c0 0x7f36ec005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:08.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.207+0000 7f3707bab700 1 -- 172.21.15.192:0/3353754222 >> 172.21.15.192:0/3353754222 conn(0x7f37000fd980 msgr2=0x7f37000ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:08.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.207+0000 7f3707bab700 1 -- 172.21.15.192:0/3353754222 shutdown_connections 2024-05-03T17:46:08.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.207+0000 7f3707bab700 1 -- 172.21.15.192:0/3353754222 wait complete. 2024-05-03T17:46:08.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.208+0000 7f3707bab700 1 Processor -- start 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.208+0000 7f3707bab700 1 -- start start 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.208+0000 7f3707bab700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec0035c0 0x7f370019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.208+0000 7f3707bab700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f370019ebe0 0x7f3700198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.208+0000 7f3707bab700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3700109de0 con 0x7f36ec0035c0 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.208+0000 7f3707bab700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3700109c60 con 0x7f370019ebe0 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.208+0000 7f3705947700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec0035c0 0x7f370019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.208+0000 7f3705146700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f370019ebe0 0x7f3700198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.208+0000 7f3705947700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec0035c0 0x7f370019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.192:34410/0 (socket says 172.21.15.192:34410) 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.208+0000 7f3705146700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f370019ebe0 0x7f3700198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:47514/0 (socket says 172.21.15.192:47514) 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.208+0000 7f3705947700 1 -- 172.21.15.192:0/2599415209 learned_addr learned my addr 172.21.15.192:0/2599415209 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f3705146700 1 -- 172.21.15.192:0/2599415209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f370019ebe0 msgr2=0x7f3700198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-03T17:46:08.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f3705146700 1 -- 172.21.15.192:0/2599415209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f370019ebe0 msgr2=0x7f3700198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T17:46:08.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f3705146700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f370019ebe0 0x7f3700198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T17:46:08.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f3705146700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f370019ebe0 0x7f3700198820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T17:46:08.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f3705947700 1 -- 172.21.15.192:0/2599415209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f370019ebe0 msgr2=0x7f3700198820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T17:46:08.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f3705947700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f370019ebe0 0x7f3700198820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:08.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f3705947700 1 -- 172.21.15.192:0/2599415209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370019f220 con 0x7f36ec0035c0 2024-05-03T17:46:08.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f3705947700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec0035c0 0x7f370019e6a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36fc001eb0 tx=0x7f36fc0050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:08.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f36f6ffd700 1 -- 172.21.15.192:0/2599415209 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f36fc004ac0 con 0x7f36ec0035c0 2024-05-03T17:46:08.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f36f6ffd700 1 -- 172.21.15.192:0/2599415209 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36fc004030 con 0x7f36ec0035c0 2024-05-03T17:46:08.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f36f6ffd700 1 -- 172.21.15.192:0/2599415209 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f36fc011b80 con 0x7f36ec0035c0 2024-05-03T17:46:08.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f3707bab700 1 -- 172.21.15.192:0/2599415209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3700198dc0 con 0x7f36ec0035c0 2024-05-03T17:46:08.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.209+0000 7f3707bab700 1 -- 172.21.15.192:0/2599415209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3700199320 con 0x7f36ec0035c0 2024-05-03T17:46:08.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.210+0000 7f36f6ffd700 1 -- 172.21.15.192:0/2599415209 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f36fc02c070 con 0x7f36ec0035c0 2024-05-03T17:46:08.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.211+0000 7f36f6ffd700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f36e403e300 0x7f36e40407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:08.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.211+0000 7f36f6ffd700 1 -- 172.21.15.192:0/2599415209 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f36fc02a070 con 0x7f36ec0035c0 2024-05-03T17:46:08.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.211+0000 7f3707bab700 1 -- 172.21.15.192:0/2599415209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37001035d0 con 0x7f36ec0035c0 2024-05-03T17:46:08.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.211+0000 7f3705146700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f36e403e300 0x7f36e40407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:08.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.211+0000 7f3705146700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f36e403e300 0x7f36e40407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009850 tx=0x7f36f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:08.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:07 smithi122 ceph-mon[29744]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:08.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:07 smithi122 ceph-mon[29744]: Deploying daemon mon.b on smithi149 2024-05-03T17:46:08.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:07 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:08.213 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.217+0000 7f36f6ffd700 1 -- 172.21.15.192:0/2599415209 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f36fc0235c0 con 0x7f36ec0035c0 2024-05-03T17:46:08.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.504+0000 7f3707bab700 1 -- 172.21.15.192:0/2599415209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3700199d80 con 0x7f36ec0035c0 2024-05-03T17:46:08.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.505+0000 7f36f6ffd700 1 -- 172.21.15.192:0/2599415209 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f36fc023360 con 0x7f36ec0035c0 2024-05-03T17:46:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:46:08.501 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","modified":"2024-05-03T17:46:01.751424Z","created":"2024-05-03T17:44:10.382958Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-03T17:46:08.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.507+0000 7f3707bab700 1 -- 172.21.15.192:0/2599415209 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f36e403e300 msgr2=0x7f36e40407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:08.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.507+0000 7f3707bab700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f36e403e300 0x7f36e40407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009850 tx=0x7f36f0008040 comp rx=0 tx=0).stop 2024-05-03T17:46:08.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.507+0000 7f3707bab700 1 -- 172.21.15.192:0/2599415209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec0035c0 msgr2=0x7f370019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:08.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.507+0000 7f3707bab700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec0035c0 0x7f370019e6a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36fc001eb0 tx=0x7f36fc0050d0 comp rx=0 tx=0).stop 2024-05-03T17:46:08.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.507+0000 7f3707bab700 1 -- 172.21.15.192:0/2599415209 shutdown_connections 2024-05-03T17:46:08.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.507+0000 7f3707bab700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f36e403e300 0x7f36e40407c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:08.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.507+0000 7f3707bab700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec0035c0 0x7f370019e6a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:08.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.507+0000 7f3707bab700 1 --2- 172.21.15.192:0/2599415209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f370019ebe0 0x7f3700198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:08.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.507+0000 7f3707bab700 1 -- 172.21.15.192:0/2599415209 >> 172.21.15.192:0/2599415209 conn(0x7f37000fd980 msgr2=0x7f37000fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:08.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.507+0000 7f3707bab700 1 -- 172.21.15.192:0/2599415209 shutdown_connections 2024-05-03T17:46:08.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:08.507+0000 7f3707bab700 1 -- 172.21.15.192:0/2599415209 wait complete. 2024-05-03T17:46:08.504 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-05-03T17:46:09.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:08 smithi192 ceph-mon[33753]: from='client.? 172.21.15.192:0/2599415209' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T17:46:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:08 smithi122 ceph-mon[29744]: from='client.? 172.21.15.192:0/2599415209' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T17:46:10.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:09 smithi192 ceph-mon[33753]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:10.127 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T17:46:10.127 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T17:46:10.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:09 smithi122 ceph-mon[29744]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:10.362 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:46:10.925 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(???) e0 preinit fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:46:10.926 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).mds e1 new map 2024-05-03T17:46:10.926 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-03T17:46:10.926 INFO:journalctl@ceph.mon.b.smithi149.stdout: e1 2024-05-03T17:46:10.926 INFO:journalctl@ceph.mon.b.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T17:46:10.926 INFO:journalctl@ceph.mon.b.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T17:46:10.926 INFO:journalctl@ceph.mon.b.smithi149.stdout: legacy client fscid: -1 2024-05-03T17:46:10.926 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-05-03T17:46:10.926 INFO:journalctl@ceph.mon.b.smithi149.stdout: No filesystems configured 2024-05-03T17:46:10.926 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-03T17:46:11.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-03T17:46:11.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-03T17:46:11.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-03T17:46:11.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-03T17:46:11.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-03T17:46:11.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Deploying cephadm binary to smithi192 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Updating smithi149:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Updating smithi149:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Added host smithi192 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:11.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3772868751' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3772868751' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Updating smithi192:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Updating smithi192:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi149:172.21.15.149=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi149:172.21.15.149=b;smithi192:172.21.15.192=c;count:3 2024-05-03T17:46:11.292 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.293 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:11.293 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:11.293 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:11.293 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.293 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:11.293 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:11.293 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Deploying daemon mon.c on smithi192 2024-05-03T17:46:11.293 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.293 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:46:11.293 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.a calling monitor election 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.c calling monitor election 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: monmap epoch 2 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:46:11.294 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: last_changed 2024-05-03T17:46:01.751424+0000 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: created 2024-05-03T17:44:10.382958+0000 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: min_mon_release 19 (squid) 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: election_strategy: 1 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: fsmap 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mgrmap e13: x(active, since 48s) 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: overall HEALTH_OK 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:11.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.296 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: Deploying daemon mon.b on smithi149 2024-05-03T17:46:11.296 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:11.296 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: from='client.? 172.21.15.192:0/2599415209' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T17:46:11.296 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:11.296 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 ceph-mon[33409]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-03T17:46:11.296 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 bash[33389]: 35163dfde5309fdefb8bd8d985ced86364fc9aca9997600920f739ebf2761ccf 2024-05-03T17:46:11.296 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:10 smithi149 systemd[1]: Started Ceph mon.b for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:46:16.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.204+0000 7fac0ffff700 1 -- 172.21.15.192:0/3593181277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabf80035c0 msgr2=0x7fabf8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:16.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.204+0000 7fac0ffff700 1 --2- 172.21.15.192:0/3593181277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabf80035c0 0x7fabf8005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fac10102d00 tx=0x7fac08009e80 comp rx=0 tx=0).stop 2024-05-03T17:46:16.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.204+0000 7fac0ffff700 1 -- 172.21.15.192:0/3593181277 shutdown_connections 2024-05-03T17:46:16.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.204+0000 7fac0ffff700 1 --2- 172.21.15.192:0/3593181277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabf80035c0 0x7fabf8005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:16.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.204+0000 7fac0ffff700 1 -- 172.21.15.192:0/3593181277 >> 172.21.15.192:0/3593181277 conn(0x7fac100fe8e0 msgr2=0x7fac10100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:16.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.204+0000 7fac0ffff700 1 -- 172.21.15.192:0/3593181277 shutdown_connections 2024-05-03T17:46:16.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.204+0000 7fac0ffff700 1 -- 172.21.15.192:0/3593181277 wait complete. 2024-05-03T17:46:16.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.205+0000 7fac0ffff700 1 Processor -- start 2024-05-03T17:46:16.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.205+0000 7fac0ffff700 1 -- start start 2024-05-03T17:46:16.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf80035c0 0x7fac1019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:16.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1019ec00 0x7fac10198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:16.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac10198dc0 0x7fac10199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:16.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0ffff700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac1010add0 con 0x7fac10198dc0 2024-05-03T17:46:16.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0ffff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fac1010ac50 con 0x7fabf80035c0 2024-05-03T17:46:16.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0ffff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fac1010af50 con 0x7fac1019ec00 2024-05-03T17:46:16.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0e7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1019ec00 0x7fac10198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0e7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1019ec00 0x7fac10198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:36194/0 (socket says 172.21.15.192:36194) 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0e7fc700 1 -- 172.21.15.192:0/431831895 learned_addr learned my addr 172.21.15.192:0/431831895 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0effd700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf80035c0 0x7fac1019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0f7fe700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac10198dc0 0x7fac10199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0effd700 1 -- 172.21.15.192:0/431831895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf80035c0 msgr2=0x7fac1019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0effd700 1 -- 172.21.15.192:0/431831895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf80035c0 msgr2=0x7fac1019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0effd700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf80035c0 0x7fac1019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0e7fc700 1 -- 172.21.15.192:0/431831895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1019ec00 msgr2=0x7fac10198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0e7fc700 1 -- 172.21.15.192:0/431831895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1019ec00 msgr2=0x7fac10198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0e7fc700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1019ec00 0x7fac10198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0effd700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf80035c0 0x7fac1019e6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.206+0000 7fac0e7fc700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1019ec00 0x7fac10198880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T17:46:16.204 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.207+0000 7fac0f7fe700 1 -- 172.21.15.192:0/431831895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1019ec00 msgr2=0x7fac10198880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T17:46:16.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.207+0000 7fac0f7fe700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1019ec00 0x7fac10198880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:16.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.207+0000 7fac0f7fe700 1 -- 172.21.15.192:0/431831895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf80035c0 msgr2=0x7fac1019e6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T17:46:16.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.207+0000 7fac0f7fe700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf80035c0 0x7fac1019e6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:16.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.207+0000 7fac0f7fe700 1 -- 172.21.15.192:0/431831895 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac101a5590 con 0x7fac10198dc0 2024-05-03T17:46:16.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.207+0000 7fac0f7fe700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac10198dc0 0x7fac10199270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fac00009990 tx=0x7fac00009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:16.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.207+0000 7fabf7fff700 1 -- 172.21.15.192:0/431831895 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00011860 con 0x7fac10198dc0 2024-05-03T17:46:16.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.207+0000 7fabf7fff700 1 -- 172.21.15.192:0/431831895 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac0001b480 con 0x7fac10198dc0 2024-05-03T17:46:16.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.207+0000 7fabf7fff700 1 -- 172.21.15.192:0/431831895 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00012670 con 0x7fac10198dc0 2024-05-03T17:46:16.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.207+0000 7fac0ffff700 1 -- 172.21.15.192:0/431831895 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac10075730 con 0x7fac10198dc0 2024-05-03T17:46:16.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.207+0000 7fac0ffff700 1 -- 172.21.15.192:0/431831895 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac10075c70 con 0x7fac10198dc0 2024-05-03T17:46:16.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.208+0000 7fabf7fff700 1 -- 172.21.15.192:0/431831895 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fac000128d0 con 0x7fac10198dc0 2024-05-03T17:46:16.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.208+0000 7fac0ffff700 1 -- 172.21.15.192:0/431831895 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac10104510 con 0x7fac10198dc0 2024-05-03T17:46:16.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.208+0000 7fabf7fff700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fabfc03e9a0 0x7fabfc040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:16.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.208+0000 7fabf7fff700 1 -- 172.21.15.192:0/431831895 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fac00057400 con 0x7fac10198dc0 2024-05-03T17:46:16.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.209+0000 7fac0effd700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fabfc03e9a0 0x7fabfc040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:16.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.209+0000 7fac0effd700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fabfc03e9a0 0x7fabfc040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fac08009160 tx=0x7fac0800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:16.210 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.214+0000 7fabf7fff700 1 -- 172.21.15.192:0/431831895 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fac0001a3e0 con 0x7fac10198dc0 2024-05-03T17:46:16.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:16.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:46:16.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: mon.a calling monitor election 2024-05-03T17:46:16.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: mon.c calling monitor election 2024-05-03T17:46:16.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: mon.b calling monitor election 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: monmap epoch 3 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: last_changed 2024-05-03T17:46:10.937396+0000 2024-05-03T17:46:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: created 2024-05-03T17:44:10.382958+0000 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: min_mon_release 19 (squid) 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: election_strategy: 1 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: 2: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.b 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: fsmap 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: mgrmap e13: x(active, since 57s) 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: overall HEALTH_OK 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:16.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: mon.a calling monitor election 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: mon.c calling monitor election 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: mon.b calling monitor election 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: monmap epoch 3 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: last_changed 2024-05-03T17:46:10.937396+0000 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: created 2024-05-03T17:44:10.382958+0000 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: min_mon_release 19 (squid) 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: election_strategy: 1 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: 2: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.b 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: fsmap 2024-05-03T17:46:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T17:46:16.369 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: mgrmap e13: x(active, since 57s) 2024-05-03T17:46:16.369 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: overall HEALTH_OK 2024-05-03T17:46:16.369 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:16.369 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:16.369 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:16.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:16.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:46:16.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: mon.a calling monitor election 2024-05-03T17:46:16.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: mon.c calling monitor election 2024-05-03T17:46:16.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:46:16.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:16.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: mon.b calling monitor election 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: monmap epoch 3 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: last_changed 2024-05-03T17:46:10.937396+0000 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: created 2024-05-03T17:44:10.382958+0000 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: min_mon_release 19 (squid) 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: election_strategy: 1 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-05-03T17:46:16.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-03T17:46:16.463 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: 2: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.b 2024-05-03T17:46:16.463 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: fsmap 2024-05-03T17:46:16.463 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T17:46:16.463 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: mgrmap e13: x(active, since 57s) 2024-05-03T17:46:16.463 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: overall HEALTH_OK 2024-05-03T17:46:16.463 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:16.463 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:16.463 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:16.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.515+0000 7fac0ffff700 1 -- 172.21.15.192:0/431831895 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fac1002e510 con 0x7fac10198dc0 2024-05-03T17:46:16.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.516+0000 7fabf7fff700 1 -- 172.21.15.192:0/431831895 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7fac0001b950 con 0x7fac10198dc0 2024-05-03T17:46:16.512 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-03T17:46:16.512 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":3,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","modified":"2024-05-03T17:46:10.937396Z","created":"2024-05-03T17:44:10.382958Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.518+0000 7fac0ffff700 1 -- 172.21.15.192:0/431831895 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fabfc03e9a0 msgr2=0x7fabfc040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.518+0000 7fac0ffff700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fabfc03e9a0 0x7fabfc040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fac08009160 tx=0x7fac0800b040 comp rx=0 tx=0).stop 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.518+0000 7fac0ffff700 1 -- 172.21.15.192:0/431831895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac10198dc0 msgr2=0x7fac10199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.518+0000 7fac0ffff700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac10198dc0 0x7fac10199270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fac00009990 tx=0x7fac00009d50 comp rx=0 tx=0).stop 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.519+0000 7fac0ffff700 1 -- 172.21.15.192:0/431831895 shutdown_connections 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.519+0000 7fac0ffff700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fabfc03e9a0 0x7fabfc040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.519+0000 7fac0ffff700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac10198dc0 0x7fac10199270 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.519+0000 7fac0ffff700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf80035c0 0x7fac1019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.519+0000 7fac0ffff700 1 --2- 172.21.15.192:0/431831895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1019ec00 0x7fac10198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.519+0000 7fac0ffff700 1 -- 172.21.15.192:0/431831895 >> 172.21.15.192:0/431831895 conn(0x7fac100fe8e0 msgr2=0x7fac100ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.519+0000 7fac0ffff700 1 -- 172.21.15.192:0/431831895 shutdown_connections 2024-05-03T17:46:16.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:16.519+0000 7fac0ffff700 1 -- 172.21.15.192:0/431831895 wait complete. 2024-05-03T17:46:16.516 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 3 2024-05-03T17:46:17.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:17.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:17.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:17 smithi192 ceph-mon[33753]: from='client.? 172.21.15.192:0/431831895' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T17:46:17.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:17.150 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-03T17:46:17.150 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-03T17:46:17.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:17.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:17.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:17 smithi149 ceph-mon[33409]: from='client.? 172.21.15.192:0/431831895' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T17:46:17.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:17.398 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:46:17.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:17.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:17.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:17 smithi122 ceph-mon[29744]: from='client.? 172.21.15.192:0/431831895' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T17:46:17.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:46:18.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:18 smithi122 ceph-mon[29744]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:18.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:18.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:18 smithi149 ceph-mon[33409]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:18.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:18.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:18.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:18 smithi192 ceph-mon[33753]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:18.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:18.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:18.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:19.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.039+0000 7f42e5125700 1 -- 172.21.15.122:0/1770653468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0101fe0 msgr2=0x7f42e01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:19.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.039+0000 7f42e5125700 1 --2- 172.21.15.122:0/1770653468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0101fe0 0x7f42e01023c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009910 tx=0x7f42d8009c20 comp rx=0 tx=0).stop 2024-05-03T17:46:19.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.039+0000 7f42e5125700 1 -- 172.21.15.122:0/1770653468 shutdown_connections 2024-05-03T17:46:19.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.039+0000 7f42e5125700 1 --2- 172.21.15.122:0/1770653468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0101fe0 0x7f42e01023c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:19.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.039+0000 7f42e5125700 1 -- 172.21.15.122:0/1770653468 >> 172.21.15.122:0/1770653468 conn(0x7f42e0078a70 msgr2=0x7f42e00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:19.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.039+0000 7f42e5125700 1 -- 172.21.15.122:0/1770653468 shutdown_connections 2024-05-03T17:46:19.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.039+0000 7f42e5125700 1 -- 172.21.15.122:0/1770653468 wait complete. 2024-05-03T17:46:19.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.040+0000 7f42e5125700 1 Processor -- start 2024-05-03T17:46:19.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.040+0000 7f42e5125700 1 -- start start 2024-05-03T17:46:19.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.040+0000 7f42e5125700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0101fe0 0x7f42e019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:19.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.040+0000 7f42e5125700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42e019dc80 0x7f42e0197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.040+0000 7f42e5125700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e0197e40 0x7f42e01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.040+0000 7f42e5125700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f42e010c260 con 0x7f42e0101fe0 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.040+0000 7f42e5125700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42e010c0e0 con 0x7f42e0197e40 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.040+0000 7f42e5125700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f42e010c3e0 con 0x7f42e019dc80 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.040+0000 7f42ded9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0101fe0 0x7f42e019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.040+0000 7f42ded9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0101fe0 0x7f42e019d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51812/0 (socket says 172.21.15.122:51812) 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.040+0000 7f42ded9d700 1 -- 172.21.15.122:0/3495759225 learned_addr learned my addr 172.21.15.122:0/3495759225 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42de59c700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42e019dc80 0x7f42e0197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42ded9d700 1 -- 172.21.15.122:0/3495759225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42e019dc80 msgr2=0x7f42e0197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42ded9d700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42e019dc80 0x7f42e0197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42ded9d700 1 -- 172.21.15.122:0/3495759225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e0197e40 msgr2=0x7f42e01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:19.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42df59e700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e0197e40 0x7f42e01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42ded9d700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e0197e40 0x7f42e01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42ded9d700 1 -- 172.21.15.122:0/3495759225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e01a4610 con 0x7f42e0101fe0 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42de59c700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42e019dc80 0x7f42e0197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42df59e700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e0197e40 0x7f42e01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42ded9d700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0101fe0 0x7f42e019d740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42d8005b40 tx=0x7f42d8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42d7fff700 1 -- 172.21.15.122:0/3495759225 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8026070 con 0x7f42e0101fe0 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42d7fff700 1 -- 172.21.15.122:0/3495759225 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42d800b6f0 con 0x7f42e0101fe0 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42e5125700 1 -- 172.21.15.122:0/3495759225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e01a4840 con 0x7f42e0101fe0 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42d7fff700 1 -- 172.21.15.122:0/3495759225 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d80038e0 con 0x7f42e0101fe0 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.041+0000 7f42e5125700 1 -- 172.21.15.122:0/3495759225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e01a4d80 con 0x7f42e0101fe0 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.042+0000 7f42d7fff700 1 -- 172.21.15.122:0/3495759225 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f42d8029070 con 0x7f42e0101fe0 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.042+0000 7f42d7fff700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f42cc03e9a0 0x7f42cc040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.042+0000 7f42d7fff700 1 -- 172.21.15.122:0/3495759225 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f42d8053500 con 0x7f42e0101fe0 2024-05-03T17:46:19.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.043+0000 7f42de59c700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f42cc03e9a0 0x7f42cc040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:19.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.043+0000 7f42e5125700 1 -- 172.21.15.122:0/3495759225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e0064b00 con 0x7f42e0101fe0 2024-05-03T17:46:19.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.043+0000 7f42de59c700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f42cc03e9a0 0x7f42cc040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f42c8007330 tx=0x7f42c80099d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:19.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.048+0000 7f42d7fff700 1 -- 172.21.15.122:0/3495759225 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f42d8030070 con 0x7f42e0101fe0 2024-05-03T17:46:19.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.299+0000 7f42e5125700 1 -- 172.21.15.122:0/3495759225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f42e01a5040 con 0x7f42e0101fe0 2024-05-03T17:46:19.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.299+0000 7f42d7fff700 1 -- 172.21.15.122:0/3495759225 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f42d8013070 con 0x7f42e0101fe0 2024-05-03T17:46:19.295 INFO:teuthology.orchestra.run.smithi122.stdout:# minimal ceph.conf for a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:46:19.295 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-05-03T17:46:19.295 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:46:19.295 INFO:teuthology.orchestra.run.smithi122.stdout: mon_host = [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-05-03T17:46:19.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.301+0000 7f42e5125700 1 -- 172.21.15.122:0/3495759225 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f42cc03e9a0 msgr2=0x7f42cc040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:19.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.301+0000 7f42e5125700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f42cc03e9a0 0x7f42cc040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f42c8007330 tx=0x7f42c80099d0 comp rx=0 tx=0).stop 2024-05-03T17:46:19.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.301+0000 7f42e5125700 1 -- 172.21.15.122:0/3495759225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0101fe0 msgr2=0x7f42e019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:19.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.301+0000 7f42e5125700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0101fe0 0x7f42e019d740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42d8005b40 tx=0x7f42d8005030 comp rx=0 tx=0).stop 2024-05-03T17:46:19.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.302+0000 7f42e5125700 1 -- 172.21.15.122:0/3495759225 shutdown_connections 2024-05-03T17:46:19.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.302+0000 7f42e5125700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f42cc03e9a0 0x7f42cc040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:19.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.302+0000 7f42e5125700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e0197e40 0x7f42e01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:19.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.302+0000 7f42e5125700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0101fe0 0x7f42e019d740 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:19.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.302+0000 7f42e5125700 1 --2- 172.21.15.122:0/3495759225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42e019dc80 0x7f42e0197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:19.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.302+0000 7f42e5125700 1 -- 172.21.15.122:0/3495759225 >> 172.21.15.122:0/3495759225 conn(0x7f42e0078a70 msgr2=0x7f42e01039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:19.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.302+0000 7f42e5125700 1 -- 172.21.15.122:0/3495759225 shutdown_connections 2024-05-03T17:46:19.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:19.302+0000 7f42e5125700 1 -- 172.21.15.122:0/3495759225 wait complete. 2024-05-03T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: Updating smithi122:/etc/ceph/ceph.conf 2024-05-03T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-03T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-03T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: Updating smithi122:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: Updating smithi192:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: Updating smithi149:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:19.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T17:46:19.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:19.655 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:19 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3495759225' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:19.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: Updating smithi122:/etc/ceph/ceph.conf 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: Updating smithi122:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: Updating smithi192:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: Updating smithi149:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:19.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T17:46:19.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:19.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:19 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3495759225' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:19.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: Updating smithi122:/etc/ceph/ceph.conf 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: Updating smithi122:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: Updating smithi192:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: Updating smithi149:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:19.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:19.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T17:46:19.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:19.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:19 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3495759225' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:19.938 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-03T17:46:19.939 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:46:19.939 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T17:46:19.964 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:46:19.964 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:46:20.028 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T17:46:20.028 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T17:46:20.057 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T17:46:20.057 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:46:20.123 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-03T17:46:20.123 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T17:46:20.155 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-03T17:46:20.155 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T17:46:20.223 INFO:tasks.cephadm:Adding mgr.x on smithi122 2024-05-03T17:46:20.223 INFO:tasks.cephadm:Adding mgr.y on smithi149 2024-05-03T17:46:20.223 INFO:tasks.cephadm:Adding mgr.z on smithi192 2024-05-03T17:46:20.223 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi122=x;smithi149=y;smithi192=z' 2024-05-03T17:46:20.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:20 smithi122 ceph-mon[29744]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:20.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:20 smithi122 ceph-mon[29744]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T17:46:20.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:20 smithi122 ceph-mon[29744]: Reconfiguring daemon mon.a on smithi122 2024-05-03T17:46:20.495 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:46:20.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:20 smithi192 ceph-mon[33753]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:20.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:20 smithi192 ceph-mon[33753]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T17:46:20.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:20 smithi192 ceph-mon[33753]: Reconfiguring daemon mon.a on smithi122 2024-05-03T17:46:20.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:20 smithi149 ceph-mon[33409]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:20.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:20 smithi149 ceph-mon[33409]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T17:46:20.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:20 smithi149 ceph-mon[33409]: Reconfiguring daemon mon.a on smithi122 2024-05-03T17:46:21.980 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.982+0000 7f6318e8b700 1 -- 172.21.15.192:0/3845234749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6314101f20 msgr2=0x7f6314102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:21.980 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.982+0000 7f6318e8b700 1 --2- 172.21.15.192:0/3845234749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6314101f20 0x7f6314102340 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6304009910 tx=0x7f6304009c20 comp rx=0 tx=0).stop 2024-05-03T17:46:21.980 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.982+0000 7f6318e8b700 1 -- 172.21.15.192:0/3845234749 shutdown_connections 2024-05-03T17:46:21.980 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.982+0000 7f6318e8b700 1 --2- 172.21.15.192:0/3845234749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6314101f20 0x7f6314102340 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:21.980 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.982+0000 7f6318e8b700 1 -- 172.21.15.192:0/3845234749 >> 172.21.15.192:0/3845234749 conn(0x7f63140fd4e0 msgr2=0x7f63140ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:21.980 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.982+0000 7f6318e8b700 1 -- 172.21.15.192:0/3845234749 shutdown_connections 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.982+0000 7f6318e8b700 1 -- 172.21.15.192:0/3845234749 wait complete. 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f6318e8b700 1 Processor -- start 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f6318e8b700 1 -- start start 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f6318e8b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6314101f20 0x7f63140ff430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f6318e8b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63140fda80 0x7f63140fdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f6318e8b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63140fe440 0x7f63141a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f6318e8b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6314107de0 con 0x7f63140fda80 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f6318e8b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6314107c60 con 0x7f63140fe440 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f6318e8b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6314107f60 con 0x7f6314101f20 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f631259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6314101f20 0x7f63140ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f631259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6314101f20 0x7f63140ff430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:36200/0 (socket says 172.21.15.192:36200) 2024-05-03T17:46:21.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f631259c700 1 -- 172.21.15.192:0/4138677007 learned_addr learned my addr 172.21.15.192:0/4138677007 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f6312d9d700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63140fe440 0x7f63141a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f6311d9b700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63140fda80 0x7f63140fdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f631259c700 1 -- 172.21.15.192:0/4138677007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63140fe440 msgr2=0x7f63141a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f631259c700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63140fe440 0x7f63141a27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f631259c700 1 -- 172.21.15.192:0/4138677007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63140fda80 msgr2=0x7f63140fdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f631259c700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63140fda80 0x7f63140fdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.983+0000 7f631259c700 1 -- 172.21.15.192:0/4138677007 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63141a2d60 con 0x7f6314101f20 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.984+0000 7f631259c700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6314101f20 0x7f63140ff430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6304005e00 tx=0x7f6304004fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.984+0000 7f63037fe700 1 -- 172.21.15.192:0/4138677007 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304027070 con 0x7f6314101f20 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.984+0000 7f63037fe700 1 -- 172.21.15.192:0/4138677007 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630400be10 con 0x7f6314101f20 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.984+0000 7f63037fe700 1 -- 172.21.15.192:0/4138677007 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630400f460 con 0x7f6314101f20 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.984+0000 7f6318e8b700 1 -- 172.21.15.192:0/4138677007 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63141a2ff0 con 0x7f6314101f20 2024-05-03T17:46:21.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.984+0000 7f6318e8b700 1 -- 172.21.15.192:0/4138677007 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63141a3460 con 0x7f6314101f20 2024-05-03T17:46:21.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.985+0000 7f63037fe700 1 -- 172.21.15.192:0/4138677007 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f630402a070 con 0x7f6314101f20 2024-05-03T17:46:21.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.985+0000 7f6318e8b700 1 -- 172.21.15.192:0/4138677007 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6314064b00 con 0x7f6314101f20 2024-05-03T17:46:21.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.985+0000 7f63037fe700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f62fc03e9a0 0x7f62fc040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:21.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.985+0000 7f63037fe700 1 -- 172.21.15.192:0/4138677007 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6304053bb0 con 0x7f6314101f20 2024-05-03T17:46:21.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.985+0000 7f6311d9b700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f62fc03e9a0 0x7f62fc040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:21.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.986+0000 7f6311d9b700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f62fc03e9a0 0x7f62fc040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6308009850 tx=0x7f6308008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:21.986 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:21.990+0000 7f63037fe700 1 -- 172.21.15.192:0/4138677007 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6304013070 con 0x7f6314101f20 2024-05-03T17:46:22.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:21 smithi149 ceph-mon[33409]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:22.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:22.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:22.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:21 smithi149 ceph-mon[33409]: Reconfiguring mon.b (monmap changed)... 2024-05-03T17:46:22.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:22.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T17:46:22.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:22.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:21 smithi149 ceph-mon[33409]: Reconfiguring daemon mon.b on smithi149 2024-05-03T17:46:22.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:21 smithi192 ceph-mon[33753]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:22.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:22.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:22.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:21 smithi192 ceph-mon[33753]: Reconfiguring mon.b (monmap changed)... 2024-05-03T17:46:22.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:22.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T17:46:22.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:22.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:21 smithi192 ceph-mon[33753]: Reconfiguring daemon mon.b on smithi149 2024-05-03T17:46:22.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.207+0000 7f6318e8b700 1 -- 172.21.15.192:0/4138677007 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi149=y;smithi192=z", "target": ["mon-mgr", ""]}) -- 0x7f6314063970 con 0x7f62fc03e9a0 2024-05-03T17:46:22.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:21 smithi122 ceph-mon[29744]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:22.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:22.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:22.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:21 smithi122 ceph-mon[29744]: Reconfiguring mon.b (monmap changed)... 2024-05-03T17:46:22.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:22.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T17:46:22.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:22.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:21 smithi122 ceph-mon[29744]: Reconfiguring daemon mon.b on smithi149 2024-05-03T17:46:22.211 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.215+0000 7f63037fe700 1 -- 172.21.15.192:0/4138677007 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6314063970 con 0x7f62fc03e9a0 2024-05-03T17:46:22.212 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mgr update... 2024-05-03T17:46:22.213 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.217+0000 7f6318e8b700 1 -- 172.21.15.192:0/4138677007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f62fc03e9a0 msgr2=0x7f62fc040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:22.214 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.217+0000 7f6318e8b700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f62fc03e9a0 0x7f62fc040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6308009850 tx=0x7f6308008040 comp rx=0 tx=0).stop 2024-05-03T17:46:22.214 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.217+0000 7f6318e8b700 1 -- 172.21.15.192:0/4138677007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6314101f20 msgr2=0x7f63140ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:22.214 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.217+0000 7f6318e8b700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6314101f20 0x7f63140ff430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6304005e00 tx=0x7f6304004fa0 comp rx=0 tx=0).stop 2024-05-03T17:46:22.214 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.218+0000 7f6318e8b700 1 -- 172.21.15.192:0/4138677007 shutdown_connections 2024-05-03T17:46:22.214 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.218+0000 7f6318e8b700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f62fc03e9a0 0x7f62fc040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:22.214 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.218+0000 7f6318e8b700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63140fe440 0x7f63141a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:22.214 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.218+0000 7f6318e8b700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6314101f20 0x7f63140ff430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:22.214 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.218+0000 7f6318e8b700 1 --2- 172.21.15.192:0/4138677007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63140fda80 0x7f63140fdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:22.214 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.218+0000 7f6318e8b700 1 -- 172.21.15.192:0/4138677007 >> 172.21.15.192:0/4138677007 conn(0x7f63140fd4e0 msgr2=0x7f631406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:22.214 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.218+0000 7f6318e8b700 1 -- 172.21.15.192:0/4138677007 shutdown_connections 2024-05-03T17:46:22.214 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:46:22.218+0000 7f6318e8b700 1 -- 172.21.15.192:0/4138677007 wait complete. 2024-05-03T17:46:22.819 DEBUG:teuthology.orchestra.run.smithi149:mgr.y> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.y.service 2024-05-03T17:46:22.822 DEBUG:teuthology.orchestra.run.smithi192:mgr.z> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.z.service 2024-05-03T17:46:22.824 INFO:tasks.cephadm:Deploying OSDs... 2024-05-03T17:46:22.825 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:46:22.825 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T17:46:22.844 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T17:46:22.844 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-05-03T17:46:22.902 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T17:46:22.902 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:22.903 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-03T17:46:22.903 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:22.903 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:22.903 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-05-03 17:45:28.163939264 +0000 2024-05-03T17:46:22.903 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-05-03 17:40:48.272434165 +0000 2024-05-03T17:46:22.903 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-05-03 17:40:48.272434165 +0000 2024-05-03T17:46:22.903 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-05-03 17:40:48.272434165 +0000 2024-05-03T17:46:22.903 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T17:46:22.970 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-05-03T17:46:22.970 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-05-03T17:46:22.971 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000206226 s, 2.5 MB/s 2024-05-03T17:46:22.972 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T17:46:23.029 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-05-03T17:46:23.085 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T17:46:23.086 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:23.086 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-03T17:46:23.086 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:23.086 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:23.086 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-05-03 17:45:28.194938559 +0000 2024-05-03T17:46:23.086 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-05-03 17:40:48.618426115 +0000 2024-05-03T17:46:23.086 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-05-03 17:40:48.618426115 +0000 2024-05-03T17:46:23.086 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-05-03 17:40:48.618426115 +0000 2024-05-03T17:46:23.086 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T17:46:23.150 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-05-03T17:46:23.150 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-05-03T17:46:23.150 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000297956 s, 1.7 MB/s 2024-05-03T17:46:23.151 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T17:46:23.208 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-05-03T17:46:23.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:23 smithi192 ceph-mon[33753]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi149=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:23.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:23 smithi192 ceph-mon[33753]: Saving service mgr spec with placement smithi122=x;smithi149=y;smithi192=z;count:3 2024-05-03T17:46:23.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:23.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:23.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:23.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:23 smithi192 ceph-mon[33753]: Reconfiguring mon.c (monmap changed)... 2024-05-03T17:46:23.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:23.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T17:46:23.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:23.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:23 smithi192 ceph-mon[33753]: Reconfiguring daemon mon.c on smithi192 2024-05-03T17:46:23.264 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T17:46:23.264 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:23.264 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-03T17:46:23.264 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:23.264 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:23.264 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-05-03 17:45:28.224937877 +0000 2024-05-03T17:46:23.265 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-05-03 17:40:48.956418252 +0000 2024-05-03T17:46:23.265 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-05-03 17:40:48.956418252 +0000 2024-05-03T17:46:23.265 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-05-03 17:40:48.956418252 +0000 2024-05-03T17:46:23.265 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T17:46:23.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:23 smithi122 ceph-mon[29744]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi149=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:23.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:23 smithi122 ceph-mon[29744]: Saving service mgr spec with placement smithi122=x;smithi149=y;smithi192=z;count:3 2024-05-03T17:46:23.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:23.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:23.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:23.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:23 smithi122 ceph-mon[29744]: Reconfiguring mon.c (monmap changed)... 2024-05-03T17:46:23.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:23.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T17:46:23.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:23.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:23 smithi122 ceph-mon[29744]: Reconfiguring daemon mon.c on smithi192 2024-05-03T17:46:23.327 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-05-03T17:46:23.327 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-05-03T17:46:23.327 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000218012 s, 2.3 MB/s 2024-05-03T17:46:23.328 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T17:46:23.384 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-05-03T17:46:23.439 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T17:46:23.439 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:23.440 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-03T17:46:23.440 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:23.440 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:23.440 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-05-03 17:45:28.254937194 +0000 2024-05-03T17:46:23.440 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-05-03 17:40:49.293410411 +0000 2024-05-03T17:46:23.440 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-05-03 17:40:49.293410411 +0000 2024-05-03T17:46:23.440 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-05-03 17:40:49.293410411 +0000 2024-05-03T17:46:23.440 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T17:46:23.501 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-05-03T17:46:23.501 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-05-03T17:46:23.502 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000238165 s, 2.1 MB/s 2024-05-03T17:46:23.503 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T17:46:23.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:23 smithi149 ceph-mon[33409]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi149=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:23.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:23 smithi149 ceph-mon[33409]: Saving service mgr spec with placement smithi122=x;smithi149=y;smithi192=z;count:3 2024-05-03T17:46:23.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:23.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:23.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:23.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:23 smithi149 ceph-mon[33409]: Reconfiguring mon.c (monmap changed)... 2024-05-03T17:46:23.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T17:46:23.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T17:46:23.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:23.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:23 smithi149 ceph-mon[33409]: Reconfiguring daemon mon.c on smithi192 2024-05-03T17:46:23.559 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T17:46:23.559 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T17:46:23.578 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T17:46:23.578 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-05-03T17:46:23.636 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T17:46:23.637 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:23.637 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-05-03T17:46:23.637 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:23.637 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:23.637 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-03 17:46:10.370642773 +0000 2024-05-03T17:46:23.637 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-03 17:41:00.182220962 +0000 2024-05-03T17:46:23.637 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-03 17:41:00.182220962 +0000 2024-05-03T17:46:23.637 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-03 17:41:00.182220962 +0000 2024-05-03T17:46:23.637 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T17:46:23.705 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-03T17:46:23.705 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-03T17:46:23.705 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000287291 s, 1.8 MB/s 2024-05-03T17:46:23.706 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T17:46:23.766 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-05-03T17:46:23.823 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T17:46:23.823 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:23.823 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-05-03T17:46:23.823 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:23.823 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:23.823 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-03 17:46:10.371642753 +0000 2024-05-03T17:46:23.823 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-03 17:41:00.527213824 +0000 2024-05-03T17:46:23.823 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-03 17:41:00.527213824 +0000 2024-05-03T17:46:23.823 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-03 17:41:00.527213824 +0000 2024-05-03T17:46:23.824 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T17:46:23.891 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-03T17:46:23.891 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-03T17:46:23.891 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000229165 s, 2.2 MB/s 2024-05-03T17:46:23.893 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T17:46:23.952 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-05-03T17:46:24.008 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T17:46:24.009 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:24.009 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-03T17:46:24.009 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:24.009 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:24.009 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-03 17:46:10.371642753 +0000 2024-05-03T17:46:24.009 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-03 17:41:00.876206602 +0000 2024-05-03T17:46:24.009 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-03 17:41:00.876206602 +0000 2024-05-03T17:46:24.009 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-03 17:41:00.876206602 +0000 2024-05-03T17:46:24.009 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T17:46:24.074 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-03T17:46:24.074 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-03T17:46:24.074 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000225945 s, 2.3 MB/s 2024-05-03T17:46:24.075 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T17:46:24.134 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-05-03T17:46:24.190 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T17:46:24.190 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:24.190 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-03T17:46:24.190 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:24.191 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:24.191 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-03 17:46:10.372642733 +0000 2024-05-03T17:46:24.191 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-03 17:41:01.223199422 +0000 2024-05-03T17:46:24.191 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-03 17:41:01.223199422 +0000 2024-05-03T17:46:24.191 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-03 17:41:01.223199422 +0000 2024-05-03T17:46:24.191 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T17:46:24.255 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-03T17:46:24.255 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-03T17:46:24.255 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000218666 s, 2.3 MB/s 2024-05-03T17:46:24.256 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T17:46:24.313 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-03T17:46:24.313 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T17:46:24.329 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T17:46:24.330 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-05-03T17:46:24.385 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T17:46:24.385 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:24.385 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-03T17:46:24.385 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:24.386 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:24.386 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-03 17:46:00.821148955 +0000 2024-05-03T17:46:24.386 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-03 17:40:58.771272773 +0000 2024-05-03T17:46:24.386 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-03 17:40:58.771272773 +0000 2024-05-03T17:46:24.386 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-03 17:40:58.771272773 +0000 2024-05-03T17:46:24.386 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T17:46:24.449 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-03T17:46:24.449 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-03T17:46:24.449 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.00025621 s, 2.0 MB/s 2024-05-03T17:46:24.451 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T17:46:24.507 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-05-03T17:46:24.561 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T17:46:24.562 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:24.562 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-03T17:46:24.562 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:24.562 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:24.562 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-03 17:46:00.821148955 +0000 2024-05-03T17:46:24.562 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-03 17:40:59.116265826 +0000 2024-05-03T17:46:24.562 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-03 17:40:59.116265826 +0000 2024-05-03T17:46:24.562 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-03 17:40:59.116265826 +0000 2024-05-03T17:46:24.562 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T17:46:24.622 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:24 smithi192 ceph-mon[33753]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:24.622 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:24 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:24.622 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:24 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:24.622 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:24 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:24.626 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-03T17:46:24.626 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-03T17:46:24.626 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000194679 s, 2.6 MB/s 2024-05-03T17:46:24.626 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T17:46:24.681 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-05-03T17:46:24.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:24 smithi122 ceph-mon[29744]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:24.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:24 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:24.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:24 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:24.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:24 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:24.735 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T17:46:24.735 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:24.735 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-03T17:46:24.735 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:24.736 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:24.736 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-03 17:46:00.821148955 +0000 2024-05-03T17:46:24.736 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-03 17:40:59.461258878 +0000 2024-05-03T17:46:24.736 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-03 17:40:59.461258878 +0000 2024-05-03T17:46:24.736 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-03 17:40:59.461258878 +0000 2024-05-03T17:46:24.736 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T17:46:24.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:24 smithi149 ceph-mon[33409]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:24.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:24 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:24.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:24 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:24.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:24 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:24.798 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-03T17:46:24.799 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-03T17:46:24.799 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000196188 s, 2.6 MB/s 2024-05-03T17:46:24.800 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T17:46:24.856 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-05-03T17:46:24.910 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T17:46:24.910 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T17:46:24.910 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-03T17:46:24.910 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T17:46:24.910 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T17:46:24.911 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-03 17:46:00.822148936 +0000 2024-05-03T17:46:24.911 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-03 17:40:59.792252212 +0000 2024-05-03T17:46:24.911 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-03 17:40:59.792252212 +0000 2024-05-03T17:46:24.911 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-03 17:40:59.792252212 +0000 2024-05-03T17:46:24.911 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T17:46:24.974 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-03T17:46:24.974 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-03T17:46:24.974 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000230404 s, 2.2 MB/s 2024-05-03T17:46:24.975 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T17:46:25.032 INFO:tasks.cephadm:Deploying osd.0 on smithi122 with /dev/vg_nvme/lv_4... 2024-05-03T17:46:25.032 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T17:46:25.264 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:46:25.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:25.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:25.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:25.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T17:46:25.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T17:46:25.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T17:46:25.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:25.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:25.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:25.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:25.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T17:46:25.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T17:46:25.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T17:46:25.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:25.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:25.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:25.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:25.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T17:46:25.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T17:46:25.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T17:46:25.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:26.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:26 smithi122 ceph-mon[29744]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:26.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:26 smithi122 ceph-mon[29744]: Deploying daemon mgr.z on smithi192 2024-05-03T17:46:26.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:26 smithi192 ceph-mon[33753]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:26.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:26 smithi192 ceph-mon[33753]: Deploying daemon mgr.z on smithi192 2024-05-03T17:46:26.616 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:26 smithi192 podman[35605]: 2024-05-03 17:46:26.485899021 +0000 UTC m=+0.018246987 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:46:26.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:26 smithi149 ceph-mon[33409]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:26.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:26 smithi149 ceph-mon[33409]: Deploying daemon mgr.z on smithi192 2024-05-03T17:46:26.944 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:26 smithi192 podman[35605]: 2024-05-03 17:46:26.794219206 +0000 UTC m=+0.326567158 container init a1483c20e32dc35b74312c92db8d909dbf503ee9d8d55e520d586c1f06df9cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-03T17:46:26.944 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:26 smithi192 podman[35605]: 2024-05-03 17:46:26.796789355 +0000 UTC m=+0.329137315 container start a1483c20e32dc35b74312c92db8d909dbf503ee9d8d55e520d586c1f06df9cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T17:46:26.944 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:26 smithi192 ceph-mgr[35624]: -- 172.21.15.192:0/2722858082 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b5a16c5a0 con 0x560b5a140800 2024-05-03T17:46:26.944 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:26 smithi192 bash[35605]: a1483c20e32dc35b74312c92db8d909dbf503ee9d8d55e520d586c1f06df9cb1 2024-05-03T17:46:26.944 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:26 smithi192 systemd[1]: Started Ceph mgr.z for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:46:27.081 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T17:46:27.081 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T17:46:27.082 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-05-03T17:46:27.082 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-05-03T17:46:27.082 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117981 s, 889 MB/s 2024-05-03T17:46:27.082 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-05-03T17:46:27.231 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:26 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:26.947+0000 7f79996c3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T17:46:27.231 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:27 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:27.133+0000 7f79996c3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T17:46:27.231 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:27 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:27.235+0000 7f79996c3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T17:46:27.684 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_4 2024-05-03T17:46:27.866 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:27 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:27.544+0000 7f79996c3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T17:46:27.866 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:27 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:27.628+0000 7f79996c3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T17:46:27.916 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:46:28.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:27 smithi122 ceph-mon[29744]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:28.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:27 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:28.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:27 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:28.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:27 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:28.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:27 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T17:46:28.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:27 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T17:46:28.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:27 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T17:46:28.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:27 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:28.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:27 smithi122 ceph-mon[29744]: Deploying daemon mgr.y on smithi149 2024-05-03T17:46:28.244 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:27 smithi192 ceph-mon[33753]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:28.244 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:27 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:28.244 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:27 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:28.244 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:27 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:28.244 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:27 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T17:46:28.244 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:27 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T17:46:28.244 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:27 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T17:46:28.244 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:27 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:28.244 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:27 smithi192 ceph-mon[33753]: Deploying daemon mgr.y on smithi149 2024-05-03T17:46:28.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:27 smithi149 ceph-mon[33409]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:28.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:27 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:28.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:27 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:28.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:27 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:28.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:27 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T17:46:28.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:27 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T17:46:28.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:27 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T17:46:28.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:27 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:28.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:27 smithi149 ceph-mon[33409]: Deploying daemon mgr.y on smithi149 2024-05-03T17:46:29.038 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:28 smithi149 systemd[1]: Starting Ceph mgr.y for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:46:29.116 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:28.860+0000 7f79996c3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T17:46:29.116 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:29.035+0000 7f79996c3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.329+0000 7f1253673700 1 -- 172.21.15.122:0/1738880277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f124c103500 msgr2=0x7f124c103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.329+0000 7f1253673700 1 --2- 172.21.15.122:0/1738880277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f124c103500 0x7f124c103920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f12480099c0 tx=0x7f1248009cd0 comp rx=0 tx=0).stop 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.329+0000 7f1253673700 1 -- 172.21.15.122:0/1738880277 shutdown_connections 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.329+0000 7f1253673700 1 --2- 172.21.15.122:0/1738880277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c1050a0 0x7f124c10b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.329+0000 7f1253673700 1 --2- 172.21.15.122:0/1738880277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f124c103500 0x7f124c103920 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.329+0000 7f1253673700 1 --2- 172.21.15.122:0/1738880277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c104700 0x7f124c104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.329+0000 7f1253673700 1 -- 172.21.15.122:0/1738880277 >> 172.21.15.122:0/1738880277 conn(0x7f124c0feaa0 msgr2=0x7f124c100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.329+0000 7f1253673700 1 -- 172.21.15.122:0/1738880277 shutdown_connections 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.329+0000 7f1253673700 1 -- 172.21.15.122:0/1738880277 wait complete. 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1253673700 1 Processor -- start 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1253673700 1 -- start start 2024-05-03T17:46:29.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1253673700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f124c103500 0x7f124c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1253673700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c104700 0x7f124c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1253673700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c1050a0 0x7f124c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1253673700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f124c10f020 con 0x7f124c103500 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1253673700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f124c10eea0 con 0x7f124c104700 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1253673700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f124c10f1a0 con 0x7f124c1050a0 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f125140f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f124c103500 0x7f124c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f125140f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f124c103500 0x7f124c1996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56354/0 (socket says 172.21.15.122:56354) 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f125140f700 1 -- 172.21.15.122:0/2588938221 learned_addr learned my addr 172.21.15.122:0/2588938221 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1251c10700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c1050a0 0x7f124c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1250c0e700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c104700 0x7f124c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f125140f700 1 -- 172.21.15.122:0/2588938221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c1050a0 msgr2=0x7f124c1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f125140f700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c1050a0 0x7f124c1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:29.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f125140f700 1 -- 172.21.15.122:0/2588938221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c104700 msgr2=0x7f124c199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f125140f700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c104700 0x7f124c199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f125140f700 1 -- 172.21.15.122:0/2588938221 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c1a1350 con 0x7f124c103500 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1250c0e700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c104700 0x7f124c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f1251c10700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c1050a0 0x7f124c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.330+0000 7f125140f700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f124c103500 0x7f124c1996f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1248009f50 tx=0x7f12480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.331+0000 7f12427fc700 1 -- 172.21.15.122:0/2588938221 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248027070 con 0x7f124c103500 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.331+0000 7f12427fc700 1 -- 172.21.15.122:0/2588938221 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12480041a0 con 0x7f124c103500 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.331+0000 7f1253673700 1 -- 172.21.15.122:0/2588938221 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c1a15e0 con 0x7f124c103500 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.331+0000 7f12427fc700 1 -- 172.21.15.122:0/2588938221 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124800f460 con 0x7f124c103500 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.331+0000 7f1253673700 1 -- 172.21.15.122:0/2588938221 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c1a1b10 con 0x7f124c103500 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.332+0000 7f12427fc700 1 -- 172.21.15.122:0/2588938221 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f124802a070 con 0x7f124c103500 2024-05-03T17:46:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.332+0000 7f1253673700 1 -- 172.21.15.122:0/2588938221 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124c06a190 con 0x7f124c103500 2024-05-03T17:46:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.332+0000 7f12427fc700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f123803e750 0x7f1238040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.332+0000 7f12427fc700 1 -- 172.21.15.122:0/2588938221 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1248053980 con 0x7f124c103500 2024-05-03T17:46:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.332+0000 7f1250c0e700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f123803e750 0x7f1238040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.332+0000 7f1250c0e700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f123803e750 0x7f1238040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f123c005fd0 tx=0x7f123c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.337+0000 7f12427fc700 1 -- 172.21.15.122:0/2588938221 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1248013070 con 0x7f124c103500 2024-05-03T17:46:29.409 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:29 smithi149 podman[34773]: 2024-05-03 17:46:29.161569722 +0000 UTC m=+0.113128437 container create 47a31b1263538a65b01fa25955cd33f701b7aaf9c20def3a7580b3585d41eb8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-03T17:46:29.409 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:29 smithi149 podman[34773]: 2024-05-03 17:46:29.066506343 +0000 UTC m=+0.018065088 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:46:29.409 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:29 smithi149 podman[34773]: 2024-05-03 17:46:29.347158233 +0000 UTC m=+0.298716951 container init 47a31b1263538a65b01fa25955cd33f701b7aaf9c20def3a7580b3585d41eb8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5) 2024-05-03T17:46:29.410 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:29 smithi149 podman[34773]: 2024-05-03 17:46:29.350106685 +0000 UTC m=+0.301665398 container start 47a31b1263538a65b01fa25955cd33f701b7aaf9c20def3a7580b3585d41eb8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, ceph=True, org.label-schema.vendor=CentOS) 2024-05-03T17:46:29.410 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:29 smithi149 ceph-mgr[34792]: -- 172.21.15.149:0/3899090776 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e04f585a0 con 0x559e04f2c800 2024-05-03T17:46:29.410 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:29 smithi149 bash[34773]: 47a31b1263538a65b01fa25955cd33f701b7aaf9c20def3a7580b3585d41eb8d 2024-05-03T17:46:29.467 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:29.211+0000 7f79996c3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T17:46:29.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:29.556+0000 7f1253673700 1 -- 172.21.15.122:0/2588938221 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f124c0638e0 con 0x7f123803e750 2024-05-03T17:46:29.705 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:29 smithi149 systemd[1]: Started Ceph mgr.y for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:46:29.705 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:29.519+0000 7fa457910200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T17:46:29.705 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:29.709+0000 7fa457910200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T17:46:29.806 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:29.508+0000 7f79996c3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T17:46:29.806 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:29.601+0000 7f79996c3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T17:46:29.806 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:29.810+0000 7f79996c3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T17:46:30.037 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:29.817+0000 7fa457910200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T17:46:30.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:29 smithi149 ceph-mon[33409]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:30.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:46:30.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:46:30.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:30.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:30.116 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:30 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:30.013+0000 7f79996c3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T17:46:30.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:29 smithi192 ceph-mon[33753]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:30.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:46:30.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:46:30.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:30.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:30.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:29 smithi122 ceph-mon[29744]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:30.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:30.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:46:30.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:46:30.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:30.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:30.522 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:30 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:30.143+0000 7fa457910200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T17:46:30.522 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:30 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:30.239+0000 7fa457910200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T17:46:30.866 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:30 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:30.647+0000 7f79996c3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T17:46:31.065 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:30 smithi122 ceph-mon[29744]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:31.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:30 smithi149 ceph-mon[33409]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:31.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:30 smithi192 ceph-mon[33753]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:31.315 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:30 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:30.874+0000 7f79996c3200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T17:46:31.616 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:31 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:31.319+0000 7f79996c3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T17:46:31.616 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:31 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:31.416+0000 7f79996c3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T17:46:31.789 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:31 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:31.501+0000 7fa457910200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T17:46:31.790 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:31 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:31.670+0000 7fa457910200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T17:46:32.116 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:31 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:31.789+0000 7f79996c3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T17:46:32.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:32.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T17:46:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T17:46:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T17:46:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:32 smithi192 ceph-mon[33753]: Reconfiguring daemon mgr.x on smithi122 2024-05-03T17:46:32.130 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:31 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:31.844+0000 7fa457910200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T17:46:32.130 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:32 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:32.134+0000 7fa457910200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T17:46:32.130 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:32.130 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.130 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.130 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.131 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.131 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:32.131 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:32.131 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.131 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T17:46:32.131 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T17:46:32.131 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T17:46:32.131 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:32.131 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:32 smithi149 ceph-mon[33409]: Reconfiguring daemon mgr.x on smithi122 2024-05-03T17:46:32.432 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:32 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:32.228+0000 7fa457910200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T17:46:32.433 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:32 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:32.437+0000 7fa457910200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T17:46:32.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:32.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:32 smithi122 ceph-mon[29744]: Reconfiguring daemon mgr.x on smithi122 2024-05-03T17:46:32.477 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:32.145+0000 7f79996c3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T17:46:32.477 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:32.237+0000 7f79996c3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T17:46:32.787 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:32 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:32.629+0000 7fa457910200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T17:46:32.866 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:32.481+0000 7f79996c3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T17:46:32.866 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:32.615+0000 7f79996c3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T17:46:33.287 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:33 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:33.238+0000 7fa457910200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T17:46:33.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:33 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3902614064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4b08628-0a29-44aa-b8b9-ed9657f12be6"}]: dispatch 2024-05-03T17:46:33.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:33 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3902614064' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4b08628-0a29-44aa-b8b9-ed9657f12be6"}]': finished 2024-05-03T17:46:33.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:33 smithi149 ceph-mon[33409]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T17:46:33.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:33.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:33 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3319562820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:46:33.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:33 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3902614064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4b08628-0a29-44aa-b8b9-ed9657f12be6"}]: dispatch 2024-05-03T17:46:33.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:33 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3902614064' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4b08628-0a29-44aa-b8b9-ed9657f12be6"}]': finished 2024-05-03T17:46:33.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:33 smithi192 ceph-mon[33753]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T17:46:33.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:33.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:33 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3319562820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:46:33.367 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:32.925+0000 7f79996c3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T17:46:33.367 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:33.041+0000 7f79996c3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T17:46:33.367 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:33.131+0000 7f79996c3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T17:46:33.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:33 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3902614064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4b08628-0a29-44aa-b8b9-ed9657f12be6"}]: dispatch 2024-05-03T17:46:33.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:33 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3902614064' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4b08628-0a29-44aa-b8b9-ed9657f12be6"}]': finished 2024-05-03T17:46:33.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:33 smithi122 ceph-mon[29744]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T17:46:33.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:33.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:33 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3319562820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:46:33.787 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:33 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:33.461+0000 7fa457910200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T17:46:34.116 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:33.735+0000 7f79996c3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T17:46:34.116 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 17:46:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z[35620]: 2024-05-03T17:46:33.951+0000 7f79996c3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T17:46:34.185 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:33 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:33.890+0000 7fa457910200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T17:46:34.185 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:33 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:33.987+0000 7fa457910200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T17:46:34.185 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:34 smithi149 ceph-mon[33409]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:34.185 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:34 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:34.185 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:34 smithi149 ceph-mon[33409]: Standby manager daemon z started 2024-05-03T17:46:34.185 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:34 smithi149 ceph-mon[33409]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T17:46:34.185 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:34 smithi149 ceph-mon[33409]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T17:46:34.186 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:34 smithi149 ceph-mon[33409]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T17:46:34.186 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:34 smithi149 ceph-mon[33409]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T17:46:34.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:34 smithi122 ceph-mon[29744]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:34.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:34 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:34.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:34 smithi122 ceph-mon[29744]: Standby manager daemon z started 2024-05-03T17:46:34.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:34 smithi122 ceph-mon[29744]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T17:46:34.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:34 smithi122 ceph-mon[29744]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T17:46:34.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:34 smithi122 ceph-mon[29744]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T17:46:34.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:34 smithi122 ceph-mon[29744]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T17:46:34.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:34.514+0000 7f12427fc700 1 -- 172.21.15.122:0/2588938221 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 14) ==== 99938+0+0 (secure 0 0 0) 0x7f1248003cf0 con 0x7f124c103500 2024-05-03T17:46:34.537 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:34 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:34.382+0000 7fa457910200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T17:46:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:34 smithi192 ceph-mon[33753]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:34 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:34 smithi192 ceph-mon[33753]: Standby manager daemon z started 2024-05-03T17:46:34.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:34 smithi192 ceph-mon[33753]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T17:46:34.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:34 smithi192 ceph-mon[33753]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T17:46:34.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:34 smithi192 ceph-mon[33753]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T17:46:34.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:34 smithi192 ceph-mon[33753]: from='mgr.? 172.21.15.192:0/3833058568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T17:46:35.037 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:34 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:34.741+0000 7fa457910200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T17:46:35.037 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:34 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:34.833+0000 7fa457910200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T17:46:35.510 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:35 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:35.078+0000 7fa457910200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T17:46:35.510 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:35 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:35.211+0000 7fa457910200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T17:46:35.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:35 smithi149 ceph-mon[33409]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:35.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:35 smithi149 ceph-mon[33409]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-03T17:46:35.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:35 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T17:46:35.788 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:35 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:35.533+0000 7fa457910200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T17:46:35.788 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:35 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:35.649+0000 7fa457910200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T17:46:35.788 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:35 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:35.740+0000 7fa457910200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T17:46:35.807 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:35 smithi122 ceph-mon[29744]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:35.807 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:35 smithi122 ceph-mon[29744]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-03T17:46:35.808 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:35 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T17:46:35.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:35 smithi192 ceph-mon[33753]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:35.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:35 smithi192 ceph-mon[33753]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-03T17:46:35.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:35 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T17:46:36.787 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:36 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:36.347+0000 7fa457910200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T17:46:36.788 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 17:46:36 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y[34788]: 2024-05-03T17:46:36.566+0000 7fa457910200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T17:46:37.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:37.524+0000 7f12427fc700 1 -- 172.21.15.122:0/2588938221 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f124800e3e0 con 0x7f124c103500 2024-05-03T17:46:37.769 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:37 smithi122 ceph-mon[29744]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:37.769 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:37 smithi122 ceph-mon[29744]: Standby manager daemon y started 2024-05-03T17:46:37.769 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:37 smithi122 ceph-mon[29744]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T17:46:37.769 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:37 smithi122 ceph-mon[29744]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T17:46:37.769 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:37 smithi122 ceph-mon[29744]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T17:46:37.769 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:37 smithi122 ceph-mon[29744]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T17:46:37.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:37 smithi149 ceph-mon[33409]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:37.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:37 smithi149 ceph-mon[33409]: Standby manager daemon y started 2024-05-03T17:46:37.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:37 smithi149 ceph-mon[33409]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T17:46:37.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:37 smithi149 ceph-mon[33409]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T17:46:37.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:37 smithi149 ceph-mon[33409]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T17:46:37.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:37 smithi149 ceph-mon[33409]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T17:46:37.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:37 smithi192 ceph-mon[33753]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:37.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:37 smithi192 ceph-mon[33753]: Standby manager daemon y started 2024-05-03T17:46:37.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:37 smithi192 ceph-mon[33753]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T17:46:37.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:37 smithi192 ceph-mon[33753]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T17:46:37.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:37 smithi192 ceph-mon[33753]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T17:46:37.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:37 smithi192 ceph-mon[33753]: from='mgr.? 172.21.15.149:0/2195414522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T17:46:38.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:38 smithi149 ceph-mon[33409]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-05-03T17:46:38.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:38 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T17:46:38.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:38 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:38.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:38 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:38.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:38 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:38.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:38 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:38.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:38 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:38.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:38 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:38.794 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:38 smithi122 ceph-mon[29744]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-05-03T17:46:38.794 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:38 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T17:46:38.794 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:38 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:38.794 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:38 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:38.794 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:38 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:38.794 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:38 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:38.794 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:38 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:38.794 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:38 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:38.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:38 smithi192 ceph-mon[33753]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-05-03T17:46:38.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:38 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T17:46:38.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:38 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:38.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:38 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:38.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:38 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:38.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:38 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:38.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:38 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:38.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:38 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:39.847 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:39 smithi122 ceph-mon[29744]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:39.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:39 smithi192 ceph-mon[33753]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:40.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:39 smithi149 ceph-mon[33409]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:40.792 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T17:46:40.792 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:40.792 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:40 smithi122 ceph-mon[29744]: Deploying daemon osd.0 on smithi122 2024-05-03T17:46:40.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:40 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T17:46:40.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:40 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:40.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:40 smithi192 ceph-mon[33753]: Deploying daemon osd.0 on smithi122 2024-05-03T17:46:41.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:40 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T17:46:41.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:40 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:41.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:40 smithi149 ceph-mon[33409]: Deploying daemon osd.0 on smithi122 2024-05-03T17:46:41.706 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:41 smithi122 ceph-mon[29744]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:41.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:41 smithi192 ceph-mon[33753]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:42.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:41 smithi149 ceph-mon[33409]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:43.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:43 smithi192 ceph-mon[33753]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:43.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:43 smithi122 ceph-mon[29744]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:44.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:43 smithi149 ceph-mon[33409]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:46.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:46 smithi122 ceph-mon[29744]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:46.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:46 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:46.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:46 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:46.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:46 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:46.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:46 smithi149 ceph-mon[33409]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:46.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:46 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:46.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:46 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:46.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:46 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:46.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:46 smithi192 ceph-mon[33753]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:46.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:46 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:46.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:46 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:46.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:46 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:48.365 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:48 smithi122 ceph-mon[29744]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:48.365 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:48 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:48.365 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:48 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:48.365 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:48 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:48.365 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:48 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:48.365 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:48 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:48.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:48 smithi149 ceph-mon[33409]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:48.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:48 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:48.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:48 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:48.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:48 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:48.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:48 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:48.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:48 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:48 smithi192 ceph-mon[33753]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:48 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:48 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:48 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:48 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:48 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:48.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.862+0000 7f12427fc700 1 -- 172.21.15.122:0/2588938221 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f124c0638e0 con 0x7f123803e750 2024-05-03T17:46:48.858 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 0 on host 'smithi122' 2024-05-03T17:46:48.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 -- 172.21.15.122:0/2588938221 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f123803e750 msgr2=0x7f1238040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:48.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f123803e750 0x7f1238040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f123c005fd0 tx=0x7f123c005ee0 comp rx=0 tx=0).stop 2024-05-03T17:46:48.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 -- 172.21.15.122:0/2588938221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f124c103500 msgr2=0x7f124c1996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:48.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f124c103500 0x7f124c1996f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1248009f50 tx=0x7f12480047b0 comp rx=0 tx=0).stop 2024-05-03T17:46:48.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 -- 172.21.15.122:0/2588938221 shutdown_connections 2024-05-03T17:46:48.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f123803e750 0x7f1238040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:48.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c1050a0 0x7f124c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:48.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f124c103500 0x7f124c1996f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:48.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 --2- 172.21.15.122:0/2588938221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c104700 0x7f124c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:48.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 -- 172.21.15.122:0/2588938221 >> 172.21.15.122:0/2588938221 conn(0x7f124c0feaa0 msgr2=0x7f124c107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:48.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 -- 172.21.15.122:0/2588938221 shutdown_connections 2024-05-03T17:46:48.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:46:48.864+0000 7f1253673700 1 -- 172.21.15.122:0/2588938221 wait complete. 2024-05-03T17:46:49.430 DEBUG:teuthology.orchestra.run.smithi122:osd.0> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.0.service 2024-05-03T17:46:49.433 INFO:tasks.cephadm:Deploying osd.1 on smithi149 with /dev/vg_nvme/lv_4... 2024-05-03T17:46:49.433 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T17:46:49.678 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T17:46:50.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:49 smithi149 ceph-mon[33409]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:50.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:50.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:50.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:50.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:49 smithi149 ceph-mon[33409]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T17:46:50.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:49 smithi122 ceph-mon[29744]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:50.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:50.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:50.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:50.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:49 smithi122 ceph-mon[29744]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T17:46:50.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:49 smithi192 ceph-mon[33753]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:50.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:50.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:50.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:46:50.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:49 smithi192 ceph-mon[33753]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T17:46:50.939 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:50 smithi149 ceph-mon[33409]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T17:46:50.939 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:50 smithi149 ceph-mon[33409]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T17:46:50.939 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:50 smithi149 ceph-mon[33409]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:46:50.939 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:50 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:50 smithi192 ceph-mon[33753]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T17:46:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:50 smithi192 ceph-mon[33753]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T17:46:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:50 smithi192 ceph-mon[33753]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:46:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:50 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:51.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:50 smithi122 ceph-mon[29744]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T17:46:51.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:50 smithi122 ceph-mon[29744]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T17:46:51.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:50 smithi122 ceph-mon[29744]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:46:51.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:51.510 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T17:46:51.510 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T17:46:51.510 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-03T17:46:51.510 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-03T17:46:51.510 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113289 s, 926 MB/s 2024-05-03T17:46:51.510 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-03T17:46:51.616 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 17:46:51 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0[42145]: 2024-05-03T17:46:51.339+0000 7f8a515cc700 -1 osd.0 0 waiting for initial osdmap 2024-05-03T17:46:51.616 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 17:46:51 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0[42145]: 2024-05-03T17:46:51.342+0000 7f8a493b7700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:46:51.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:51 smithi122 ceph-mon[29744]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:51.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:51 smithi122 ceph-mon[29744]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:46:51.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:51 smithi122 ceph-mon[29744]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T17:46:51.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:51 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:51.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:51 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:51.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:51 smithi122 ceph-mon[29744]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' 2024-05-03T17:46:52.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:51 smithi192 ceph-mon[33753]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:52.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:51 smithi192 ceph-mon[33753]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:46:52.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:51 smithi192 ceph-mon[33753]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T17:46:52.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:51 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:52.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:51 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:52.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:51 smithi192 ceph-mon[33753]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' 2024-05-03T17:46:52.132 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:51 smithi149 ceph-mon[33409]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:52.132 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:51 smithi149 ceph-mon[33409]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:46:52.132 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:51 smithi149 ceph-mon[33409]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T17:46:52.132 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:51 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:52.133 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:51 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:52.133 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:51 smithi149 ceph-mon[33409]: from='osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610]' entity='osd.0' 2024-05-03T17:46:52.160 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-05-03T17:46:52.406 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T17:46:53.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:46:53.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi122 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi122 to 17088M 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610] boot 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:53.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:53.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:46:53.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:46:53.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi122 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi122 to 17088M 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610] boot 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:53.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:46:53.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:46:53.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:53.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi122 2024-05-03T17:46:53.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:46:53.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi122 to 17088M 2024-05-03T17:46:53.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:53.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: osd.0 [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610] boot 2024-05-03T17:46:53.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T17:46:53.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:46:53.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T17:46:53.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:46:53.891 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/190602026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d28104700 msgr2=0x7f2d28104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:53.891 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 --2- 172.21.15.149:0/190602026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d28104700 0x7f2d28104b60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2d140099c0 tx=0x7f2d14009cd0 comp rx=0 tx=0).stop 2024-05-03T17:46:53.891 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/190602026 shutdown_connections 2024-05-03T17:46:53.891 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 --2- 172.21.15.149:0/190602026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d281050a0 0x7f2d2810b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:53.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 --2- 172.21.15.149:0/190602026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d28103500 0x7f2d28103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:53.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 --2- 172.21.15.149:0/190602026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d28104700 0x7f2d28104b60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:53.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/190602026 >> 172.21.15.149:0/190602026 conn(0x7f2d280feaa0 msgr2=0x7f2d28100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:46:53.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/190602026 shutdown_connections 2024-05-03T17:46:53.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/190602026 wait complete. 2024-05-03T17:46:53.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 Processor -- start 2024-05-03T17:46:53.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 -- start start 2024-05-03T17:46:53.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d28103500 0x7f2d28199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:53.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d28104700 0x7f2d28199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:53.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.892+0000 7f2d2ec6d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d281050a0 0x7f2d281a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:53.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ec6d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2810f020 con 0x7f2d28104700 2024-05-03T17:46:53.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ec6d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2810eea0 con 0x7f2d28103500 2024-05-03T17:46:53.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ec6d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2810f1a0 con 0x7f2d281050a0 2024-05-03T17:46:53.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ca09700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d28103500 0x7f2d28199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:53.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ca09700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d28103500 0x7f2d28199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:49086/0 (socket says 172.21.15.149:49086) 2024-05-03T17:46:53.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ca09700 1 -- 172.21.15.149:0/3625557112 learned_addr learned my addr 172.21.15.149:0/3625557112 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T17:46:53.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d27fff700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d28104700 0x7f2d28199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:53.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ca09700 1 -- 172.21.15.149:0/3625557112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d281050a0 msgr2=0x7f2d281a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:53.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2d20a700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d281050a0 0x7f2d281a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:53.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ca09700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d281050a0 0x7f2d281a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:53.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ca09700 1 -- 172.21.15.149:0/3625557112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d28104700 msgr2=0x7f2d28199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:46:53.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ca09700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d28104700 0x7f2d28199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:46:53.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ca09700 1 -- 172.21.15.149:0/3625557112 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d281a1380 con 0x7f2d28103500 2024-05-03T17:46:53.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2d20a700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d281050a0 0x7f2d281a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:46:53.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ca09700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d28103500 0x7f2d28199720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b500 tx=0x7f2d1c00b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:53.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d25ffb700 1 -- 172.21.15.149:0/3625557112 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c015020 con 0x7f2d28103500 2024-05-03T17:46:53.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d25ffb700 1 -- 172.21.15.149:0/3625557112 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d1c011ce0 con 0x7f2d28103500 2024-05-03T17:46:53.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d25ffb700 1 -- 172.21.15.149:0/3625557112 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c010d50 con 0x7f2d28103500 2024-05-03T17:46:53.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.893+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/3625557112 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d281a1670 con 0x7f2d28103500 2024-05-03T17:46:53.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.894+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/3625557112 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d281a1bb0 con 0x7f2d28103500 2024-05-03T17:46:53.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.894+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/3625557112 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2806a190 con 0x7f2d28103500 2024-05-03T17:46:53.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.896+0000 7f2d25ffb700 1 -- 172.21.15.149:0/3625557112 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f2d1c018020 con 0x7f2d28103500 2024-05-03T17:46:53.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.896+0000 7f2d25ffb700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2d180b27f0 0x7f2d180b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:46:53.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.896+0000 7f2d25ffb700 1 -- 172.21.15.149:0/3625557112 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f2d1c0dff30 con 0x7f2d28103500 2024-05-03T17:46:53.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.896+0000 7f2d27fff700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2d180b27f0 0x7f2d180b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:46:53.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.897+0000 7f2d27fff700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2d180b27f0 0x7f2d180b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2d14000c00 tx=0x7f2d14005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:46:53.897 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:53.901+0000 7f2d25ffb700 1 -- 172.21.15.149:0/3625557112 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2d1c01f020 con 0x7f2d28103500 2024-05-03T17:46:54.109 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:46:54.113+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/3625557112 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2d280638e0 con 0x7f2d180b27f0 2024-05-03T17:46:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:54 smithi149 ceph-mon[33409]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:54 smithi149 ceph-mon[33409]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T17:46:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:46:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:46:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:54.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:54 smithi192 ceph-mon[33753]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:54.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:54 smithi192 ceph-mon[33753]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T17:46:54.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:46:54.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:46:54.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:54.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:54 smithi122 ceph-mon[29744]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T17:46:54.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:54 smithi122 ceph-mon[29744]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T17:46:54.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:46:54.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:46:54.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:46:55.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:55 smithi192 ceph-mon[33753]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:55.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:55 smithi149 ceph-mon[33409]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:55.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:55 smithi122 ceph-mon[29744]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:46:56.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:56 smithi192 ceph-mon[33753]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:46:56.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:56 smithi122 ceph-mon[29744]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:46:56.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:56 smithi149 ceph-mon[33409]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:46:57.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:57 smithi122 ceph-mon[29744]: from='client.? 172.21.15.149:0/1295866882' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba550787-ce76-4944-8ebd-97e19590bb3b"}]: dispatch 2024-05-03T17:46:57.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:57 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba550787-ce76-4944-8ebd-97e19590bb3b"}]: dispatch 2024-05-03T17:46:57.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:57 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba550787-ce76-4944-8ebd-97e19590bb3b"}]': finished 2024-05-03T17:46:57.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:57 smithi122 ceph-mon[29744]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T17:46:57.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:46:57.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:57 smithi122 ceph-mon[29744]: from='client.? 172.21.15.149:0/2848752849' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:46:57.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:57 smithi149 ceph-mon[33409]: from='client.? 172.21.15.149:0/1295866882' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba550787-ce76-4944-8ebd-97e19590bb3b"}]: dispatch 2024-05-03T17:46:57.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:57 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba550787-ce76-4944-8ebd-97e19590bb3b"}]: dispatch 2024-05-03T17:46:57.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:57 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba550787-ce76-4944-8ebd-97e19590bb3b"}]': finished 2024-05-03T17:46:57.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:57 smithi149 ceph-mon[33409]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T17:46:57.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:46:57.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:57 smithi149 ceph-mon[33409]: from='client.? 172.21.15.149:0/2848752849' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:46:57.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:57 smithi192 ceph-mon[33753]: from='client.? 172.21.15.149:0/1295866882' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba550787-ce76-4944-8ebd-97e19590bb3b"}]: dispatch 2024-05-03T17:46:57.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:57 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba550787-ce76-4944-8ebd-97e19590bb3b"}]: dispatch 2024-05-03T17:46:57.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:57 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba550787-ce76-4944-8ebd-97e19590bb3b"}]': finished 2024-05-03T17:46:57.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:57 smithi192 ceph-mon[33753]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T17:46:57.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:46:57.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:57 smithi192 ceph-mon[33753]: from='client.? 172.21.15.149:0/2848752849' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:46:58.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:46:58 smithi122 ceph-mon[29744]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:46:58.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:46:58 smithi149 ceph-mon[33409]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:46:58.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:46:58 smithi192 ceph-mon[33753]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:00.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:00 smithi149 ceph-mon[33409]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:00 smithi122 ceph-mon[29744]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:00.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:00 smithi192 ceph-mon[33753]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:02.647 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:02 smithi149 ceph-mon[33409]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:02.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:02 smithi122 ceph-mon[29744]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:02.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:02 smithi192 ceph-mon[33753]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:04.495 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:04 smithi149 ceph-mon[33409]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:04.495 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T17:47:04.495 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:04.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:04 smithi122 ceph-mon[29744]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:04.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T17:47:04.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:04.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:04 smithi192 ceph-mon[33753]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:04.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T17:47:04.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:05.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:05 smithi149 ceph-mon[33409]: Deploying daemon osd.1 on smithi149 2024-05-03T17:47:05.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:05 smithi122 ceph-mon[29744]: Deploying daemon osd.1 on smithi149 2024-05-03T17:47:05.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:05 smithi192 ceph-mon[33753]: Deploying daemon osd.1 on smithi149 2024-05-03T17:47:06.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:06 smithi149 ceph-mon[33409]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:06.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:06 smithi122 ceph-mon[29744]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:06.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:06 smithi192 ceph-mon[33753]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:08.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:08 smithi149 ceph-mon[33409]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:08.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:08 smithi122 ceph-mon[29744]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:08.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:08 smithi192 ceph-mon[33753]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:09.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:09 smithi149 ceph-mon[33409]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:09.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:09 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:09.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:09 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:09.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:09 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:10.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:09 smithi192 ceph-mon[33753]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:10.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:09 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:10.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:09 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:10.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:09 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:10.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:09 smithi122 ceph-mon[29744]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:10.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:09 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:10.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:09 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:10.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:09 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:12.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:11 smithi192 ceph-mon[33753]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:12.157 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:11 smithi149 ceph-mon[33409]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:12.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:11 smithi122 ceph-mon[29744]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 1 on host 'smithi149' 2024-05-03T17:47:12.224 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.228+0000 7f2d25ffb700 1 -- 172.21.15.149:0/3625557112 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2d280638e0 con 0x7f2d180b27f0 2024-05-03T17:47:12.227 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.231+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/3625557112 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2d180b27f0 msgr2=0x7f2d180b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:47:12.227 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.231+0000 7f2d2ec6d700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2d180b27f0 0x7f2d180b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2d14000c00 tx=0x7f2d14005fb0 comp rx=0 tx=0).stop 2024-05-03T17:47:12.227 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.231+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/3625557112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d28103500 msgr2=0x7f2d28199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:47:12.227 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.231+0000 7f2d2ec6d700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d28103500 0x7f2d28199720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b500 tx=0x7f2d1c00b8c0 comp rx=0 tx=0).stop 2024-05-03T17:47:12.227 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.231+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/3625557112 shutdown_connections 2024-05-03T17:47:12.227 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.231+0000 7f2d2ec6d700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2d180b27f0 0x7f2d180b4cb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:12.227 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.231+0000 7f2d2ec6d700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d281050a0 0x7f2d281a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:12.227 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.231+0000 7f2d2ec6d700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d28103500 0x7f2d28199720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:12.227 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.231+0000 7f2d2ec6d700 1 --2- 172.21.15.149:0/3625557112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d28104700 0x7f2d28199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:12.227 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.231+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/3625557112 >> 172.21.15.149:0/3625557112 conn(0x7f2d280feaa0 msgr2=0x7f2d28107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:47:12.228 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.232+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/3625557112 shutdown_connections 2024-05-03T17:47:12.228 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:47:12.232+0000 7f2d2ec6d700 1 -- 172.21.15.149:0/3625557112 wait complete. 2024-05-03T17:47:12.837 DEBUG:teuthology.orchestra.run.smithi149:osd.1> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.1.service 2024-05-03T17:47:12.839 INFO:tasks.cephadm:Deploying osd.2 on smithi192 with /dev/vg_nvme/lv_4... 2024-05-03T17:47:12.840 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T17:47:13.085 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:47:13.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:13 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:13.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:13 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:13.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:13 smithi122 ceph-mon[29744]: from='osd.1 [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T17:47:13.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:13 smithi122 ceph-mon[29744]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T17:47:13.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:13 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:13.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:13 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:13.477 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:13 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:13.478 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:13 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:13.478 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:13 smithi192 ceph-mon[33753]: from='osd.1 [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T17:47:13.478 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:13 smithi192 ceph-mon[33753]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T17:47:13.478 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:13 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:13.478 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:13 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:13.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:13 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:13.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:13 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:13.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:13 smithi149 ceph-mon[33409]: from='osd.1 [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T17:47:13.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:13 smithi149 ceph-mon[33409]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T17:47:13.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:13 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:13.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:13 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:14.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:14 smithi149 ceph-mon[33409]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:14.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:14 smithi149 ceph-mon[33409]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T17:47:14.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:14 smithi149 ceph-mon[33409]: from='osd.1 [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:47:14.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:14 smithi149 ceph-mon[33409]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T17:47:14.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:14 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:14.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:14 smithi149 ceph-mon[33409]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:47:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:14 smithi192 ceph-mon[33753]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:14 smithi192 ceph-mon[33753]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T17:47:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:14 smithi192 ceph-mon[33753]: from='osd.1 [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:47:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:14 smithi192 ceph-mon[33753]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T17:47:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:14 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:14 smithi192 ceph-mon[33753]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:47:14.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:14 smithi122 ceph-mon[29744]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:14.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:14 smithi122 ceph-mon[29744]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T17:47:14.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:14 smithi122 ceph-mon[29744]: from='osd.1 [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:47:14.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:14 smithi122 ceph-mon[29744]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T17:47:14.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:14 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:14.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:14 smithi122 ceph-mon[29744]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:47:14.860 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 17:47:14 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1[38021]: 2024-05-03T17:47:14.672+0000 7f9a64f06700 -1 osd.1 0 waiting for initial osdmap 2024-05-03T17:47:14.860 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 17:47:14 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1[38021]: 2024-05-03T17:47:14.676+0000 7f9a5ccf1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:47:14.987 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T17:47:14.988 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T17:47:14.988 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-05-03T17:47:14.988 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-05-03T17:47:14.988 INFO:teuthology.orchestra.run.smithi192.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112423 s, 933 MB/s 2024-05-03T17:47:14.988 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 2024-05-03T17:47:14.988 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-05-03T17:47:15.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:15 smithi122 ceph-mon[29744]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:47:15.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:15 smithi122 ceph-mon[29744]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T17:47:15.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:15.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:15.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:15 smithi122 ceph-mon[29744]: from='osd.1 ' entity='osd.1' 2024-05-03T17:47:15.504 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:15 smithi192 ceph-mon[33753]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:47:15.505 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:15 smithi192 ceph-mon[33753]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T17:47:15.505 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:15.505 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:15.505 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:15 smithi192 ceph-mon[33753]: from='osd.1 ' entity='osd.1' 2024-05-03T17:47:15.533 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2024-05-03T17:47:15.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:15 smithi149 ceph-mon[33409]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:47:15.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:15 smithi149 ceph-mon[33409]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T17:47:15.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:15.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:15.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:15 smithi149 ceph-mon[33409]: from='osd.1 ' entity='osd.1' 2024-05-03T17:47:15.772 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:47:16.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:16 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:47:16.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:16 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:47:16.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:16 smithi122 ceph-mon[29744]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:16.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:16.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:16 smithi122 ceph-mon[29744]: osd.1 [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302] boot 2024-05-03T17:47:16.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:16 smithi122 ceph-mon[29744]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T17:47:16.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:16.475 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:16 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:47:16.475 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:16 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:47:16.475 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:16 smithi149 ceph-mon[33409]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:16.475 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:16.475 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:16 smithi149 ceph-mon[33409]: osd.1 [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302] boot 2024-05-03T17:47:16.475 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:16 smithi149 ceph-mon[33409]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T17:47:16.475 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:16.494 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:16 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:47:16.494 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:16 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:47:16.494 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:16 smithi192 ceph-mon[33753]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T17:47:16.494 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:16.494 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:16 smithi192 ceph-mon[33753]: osd.1 [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302] boot 2024-05-03T17:47:16.494 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:16 smithi192 ceph-mon[33753]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T17:47:16.494 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T17:47:17.306 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.307+0000 7fd78537c700 1 -- 172.21.15.192:0/306651539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd780103fc0 msgr2=0x7fd78010a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:47:17.306 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.307+0000 7fd78537c700 1 --2- 172.21.15.192:0/306651539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd780103fc0 0x7fd78010a870 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd770009990 tx=0x7fd770009ca0 comp rx=0 tx=0).stop 2024-05-03T17:47:17.306 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 -- 172.21.15.192:0/306651539 shutdown_connections 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 --2- 172.21.15.192:0/306651539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd780103fc0 0x7fd78010a870 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 --2- 172.21.15.192:0/306651539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd780102420 0x7fd780102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 --2- 172.21.15.192:0/306651539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd780103620 0x7fd780103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 -- 172.21.15.192:0/306651539 >> 172.21.15.192:0/306651539 conn(0x7fd7800fda00 msgr2=0x7fd7800ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 -- 172.21.15.192:0/306651539 shutdown_connections 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 -- 172.21.15.192:0/306651539 wait complete. 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 Processor -- start 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 -- start start 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd780102420 0x7fd780199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd780103620 0x7fd780199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.308+0000 7fd78537c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd780103fc0 0x7fd7801a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd78537c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd78010de90 con 0x7fd780103620 2024-05-03T17:47:17.307 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd78537c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd78010dd10 con 0x7fd780103fc0 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd78537c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd78010e010 con 0x7fd780102420 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd780102420 0x7fd780199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd780102420 0x7fd780199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:49634/0 (socket says 172.21.15.192:49634) 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77effd700 1 -- 172.21.15.192:0/894399353 learned_addr learned my addr 172.21.15.192:0/894399353 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77e7fc700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd780103620 0x7fd780199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77f7fe700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd780103fc0 0x7fd7801a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77effd700 1 -- 172.21.15.192:0/894399353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd780103fc0 msgr2=0x7fd7801a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77effd700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd780103fc0 0x7fd7801a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77effd700 1 -- 172.21.15.192:0/894399353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd780103620 msgr2=0x7fd780199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77effd700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd780103620 0x7fd780199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77effd700 1 -- 172.21.15.192:0/894399353 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7801a0f90 con 0x7fd780102420 2024-05-03T17:47:17.308 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77f7fe700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd780103fc0 0x7fd7801a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:47:17.309 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd77effd700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd780102420 0x7fd780199330 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd77800add0 tx=0x7fd77800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:47:17.309 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd767fff700 1 -- 172.21.15.192:0/894399353 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778017020 con 0x7fd780102420 2024-05-03T17:47:17.309 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd767fff700 1 -- 172.21.15.192:0/894399353 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd77800c940 con 0x7fd780102420 2024-05-03T17:47:17.309 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd767fff700 1 -- 172.21.15.192:0/894399353 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778004280 con 0x7fd780102420 2024-05-03T17:47:17.309 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd78537c700 1 -- 172.21.15.192:0/894399353 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7801a1280 con 0x7fd780102420 2024-05-03T17:47:17.309 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.309+0000 7fd78537c700 1 -- 172.21.15.192:0/894399353 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7801a1810 con 0x7fd780102420 2024-05-03T17:47:17.309 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.310+0000 7fd78537c700 1 -- 172.21.15.192:0/894399353 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd78006a190 con 0x7fd780102420 2024-05-03T17:47:17.310 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.312+0000 7fd767fff700 1 -- 172.21.15.192:0/894399353 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fd77800caf0 con 0x7fd780102420 2024-05-03T17:47:17.310 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.312+0000 7fd767fff700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd7680b2870 0x7fd7680b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:47:17.310 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.312+0000 7fd767fff700 1 -- 172.21.15.192:0/894399353 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fd7780e02d0 con 0x7fd780102420 2024-05-03T17:47:17.310 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.313+0000 7fd77e7fc700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd7680b2870 0x7fd7680b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:47:17.310 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.313+0000 7fd77e7fc700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd7680b2870 0x7fd7680b4d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd77400ac50 tx=0x7fd77400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:47:17.312 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.316+0000 7fd767fff700 1 -- 172.21.15.192:0/894399353 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd778021020 con 0x7fd780102420 2024-05-03T17:47:17.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:17.527+0000 7fd78537c700 1 -- 172.21.15.192:0/894399353 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd7800638e0 con 0x7fd7680b2870 2024-05-03T17:47:17.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:17.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi149 2024-05-03T17:47:17.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:47:17.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi149 to 5816M 2024-05-03T17:47:17.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:17.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:17.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:17.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:17.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:17.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T17:47:17.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:47:17.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:47:17.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:17.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:17.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi149 2024-05-03T17:47:17.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi149 to 5816M 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:47:17.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:47:17.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi149 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi149 to 5816M 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:17.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:17.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:17.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:17.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T17:47:17.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:47:17.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:47:17.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:18.745 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:18 smithi192 ceph-mon[33753]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:47:18.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:18 smithi122 ceph-mon[29744]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:47:19.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:18 smithi149 ceph-mon[33409]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:47:19.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:19 smithi149 ceph-mon[33409]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:19.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:19 smithi192 ceph-mon[33753]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:19.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:19 smithi122 ceph-mon[29744]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:20.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:20 smithi192 ceph-mon[33753]: from='client.? 172.21.15.192:0/2871135101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35ae7c47-d710-47c5-b0ec-91fa12b5be07"}]: dispatch 2024-05-03T17:47:20.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:20 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35ae7c47-d710-47c5-b0ec-91fa12b5be07"}]: dispatch 2024-05-03T17:47:20.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:20 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35ae7c47-d710-47c5-b0ec-91fa12b5be07"}]': finished 2024-05-03T17:47:20.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:20 smithi192 ceph-mon[33753]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T17:47:20.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:20 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:20.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:20 smithi122 ceph-mon[29744]: from='client.? 172.21.15.192:0/2871135101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35ae7c47-d710-47c5-b0ec-91fa12b5be07"}]: dispatch 2024-05-03T17:47:20.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:20 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35ae7c47-d710-47c5-b0ec-91fa12b5be07"}]: dispatch 2024-05-03T17:47:20.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:20 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35ae7c47-d710-47c5-b0ec-91fa12b5be07"}]': finished 2024-05-03T17:47:20.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:20 smithi122 ceph-mon[29744]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T17:47:20.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:20 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:21.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:20 smithi149 ceph-mon[33409]: from='client.? 172.21.15.192:0/2871135101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35ae7c47-d710-47c5-b0ec-91fa12b5be07"}]: dispatch 2024-05-03T17:47:21.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:20 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35ae7c47-d710-47c5-b0ec-91fa12b5be07"}]: dispatch 2024-05-03T17:47:21.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:20 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35ae7c47-d710-47c5-b0ec-91fa12b5be07"}]': finished 2024-05-03T17:47:21.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:20 smithi149 ceph-mon[33409]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T17:47:21.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:20 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:21.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:21 smithi192 ceph-mon[33753]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:21.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:21 smithi192 ceph-mon[33753]: from='client.? 172.21.15.192:0/2480807034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:47:21.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:21 smithi122 ceph-mon[29744]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:21.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:21 smithi122 ceph-mon[29744]: from='client.? 172.21.15.192:0/2480807034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:47:22.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:21 smithi149 ceph-mon[33409]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:22.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:21 smithi149 ceph-mon[33409]: from='client.? 172.21.15.192:0/2480807034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:47:23.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:23 smithi192 ceph-mon[33753]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:23.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:23 smithi122 ceph-mon[29744]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:24.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:23 smithi149 ceph-mon[33409]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:25.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:25 smithi192 ceph-mon[33753]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:25.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:25 smithi122 ceph-mon[29744]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:26.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:25 smithi149 ceph-mon[33409]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:27.737 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:27 smithi192 ceph-mon[33753]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:27.737 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:27 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T17:47:27.737 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:27 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:27.737 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:27 smithi192 ceph-mon[33753]: Deploying daemon osd.2 on smithi192 2024-05-03T17:47:27.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:27 smithi122 ceph-mon[29744]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:27.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:27 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T17:47:27.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:27 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:27.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:27 smithi122 ceph-mon[29744]: Deploying daemon osd.2 on smithi192 2024-05-03T17:47:28.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:27 smithi149 ceph-mon[33409]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:28.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:27 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T17:47:28.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:27 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:28.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:27 smithi149 ceph-mon[33409]: Deploying daemon osd.2 on smithi192 2024-05-03T17:47:29.799 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:29 smithi192 ceph-mon[33753]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:29.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:29 smithi122 ceph-mon[29744]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:30.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:29 smithi149 ceph-mon[33409]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:31.798 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:31 smithi192 ceph-mon[33753]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:31.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:31 smithi122 ceph-mon[29744]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:32.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:31 smithi149 ceph-mon[33409]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:33.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:33 smithi192 ceph-mon[33753]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:33.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:33.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:33.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:33.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:33 smithi122 ceph-mon[29744]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:33.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:33.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:33.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:34.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:33 smithi149 ceph-mon[33409]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:34.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:34.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:34.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:36.088 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 2 on host 'smithi192' 2024-05-03T17:47:36.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.091+0000 7fd767fff700 1 -- 172.21.15.192:0/894399353 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd7800638e0 con 0x7fd7680b2870 2024-05-03T17:47:36.090 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 -- 172.21.15.192:0/894399353 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd7680b2870 msgr2=0x7fd7680b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:47:36.090 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd7680b2870 0x7fd7680b4d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd77400ac50 tx=0x7fd77400a380 comp rx=0 tx=0).stop 2024-05-03T17:47:36.090 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 -- 172.21.15.192:0/894399353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd780102420 msgr2=0x7fd780199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:47:36.090 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd780102420 0x7fd780199330 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd77800add0 tx=0x7fd77800c330 comp rx=0 tx=0).stop 2024-05-03T17:47:36.090 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 -- 172.21.15.192:0/894399353 shutdown_connections 2024-05-03T17:47:36.090 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd7680b2870 0x7fd7680b4d30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:36.090 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd780103fc0 0x7fd7801a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:36.090 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd780102420 0x7fd780199330 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:36.090 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 --2- 172.21.15.192:0/894399353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd780103620 0x7fd780199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:36.090 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 -- 172.21.15.192:0/894399353 >> 172.21.15.192:0/894399353 conn(0x7fd7800fda00 msgr2=0x7fd780100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:47:36.090 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 -- 172.21.15.192:0/894399353 shutdown_connections 2024-05-03T17:47:36.091 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:47:36.094+0000 7fd78537c700 1 -- 172.21.15.192:0/894399353 wait complete. 2024-05-03T17:47:36.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:36 smithi192 ceph-mon[33753]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:36.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:36 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:36 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:36 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:36.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:36 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:36.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:36 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:36 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:36 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:36 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:36.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:36 smithi149 ceph-mon[33409]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:36.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:36 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:36 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:36 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:36.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:36 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:36.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:36 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:36 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:36 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:36 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:36.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:36 smithi122 ceph-mon[29744]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:36.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:36 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:36 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:36 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:36.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:36 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:36.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:36 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:36 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:36 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:36.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:36 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:36.773 DEBUG:teuthology.orchestra.run.smithi192:osd.2> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.2.service 2024-05-03T17:47:36.775 INFO:tasks.cephadm:Deploying osd.3 on smithi122 with /dev/vg_nvme/lv_3... 2024-05-03T17:47:36.775 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T17:47:37.027 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:47:37.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:37 smithi122 ceph-mon[29744]: from='osd.2 [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T17:47:37.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:37 smithi122 ceph-mon[29744]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T17:47:37.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:37 smithi149 ceph-mon[33409]: from='osd.2 [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T17:47:37.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:37 smithi149 ceph-mon[33409]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T17:47:37.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:37 smithi192 ceph-mon[33753]: from='osd.2 [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T17:47:37.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:37 smithi192 ceph-mon[33753]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T17:47:38.430 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:38 smithi122 ceph-mon[29744]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:38.430 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:38 smithi122 ceph-mon[29744]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T17:47:38.430 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:38 smithi122 ceph-mon[29744]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T17:47:38.430 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:38 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:38.430 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:38 smithi122 ceph-mon[29744]: from='osd.2 [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:47:38.430 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:38 smithi122 ceph-mon[29744]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:47:38.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:38 smithi192 ceph-mon[33753]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:38.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:38 smithi192 ceph-mon[33753]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T17:47:38.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:38 smithi192 ceph-mon[33753]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T17:47:38.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:38 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:38.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:38 smithi192 ceph-mon[33753]: from='osd.2 [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:47:38.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:38 smithi192 ceph-mon[33753]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:47:38.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:38 smithi149 ceph-mon[33409]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:38.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:38 smithi149 ceph-mon[33409]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T17:47:38.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:38 smithi149 ceph-mon[33409]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T17:47:38.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:38 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:38.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:38 smithi149 ceph-mon[33409]: from='osd.2 [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:47:38.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:38 smithi149 ceph-mon[33409]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:47:39.012 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 17:47:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2[38922]: 2024-05-03T17:47:38.727+0000 7f156d999700 -1 osd.2 0 waiting for initial osdmap 2024-05-03T17:47:39.012 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 17:47:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2[38922]: 2024-05-03T17:47:38.732+0000 7f1564f83700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:47:39.049 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T17:47:39.050 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T17:47:39.050 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-05-03T17:47:39.050 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-05-03T17:47:39.050 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.010926 s, 960 MB/s 2024-05-03T17:47:39.050 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-05-03T17:47:39.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:39 smithi192 ceph-mon[33753]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:47:39.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:39 smithi192 ceph-mon[33753]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T17:47:39.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:39 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:39.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:39 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:39.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:39 smithi192 ceph-mon[33753]: from='osd.2 ' entity='osd.2' 2024-05-03T17:47:39.669 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:39 smithi122 ceph-mon[29744]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:47:39.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:39 smithi122 ceph-mon[29744]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T17:47:39.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:39 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:39.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:39 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:39.670 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:39 smithi122 ceph-mon[29744]: from='osd.2 ' entity='osd.2' 2024-05-03T17:47:39.697 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_3 2024-05-03T17:47:39.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:39 smithi149 ceph-mon[33409]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:47:39.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:39 smithi149 ceph-mon[33409]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T17:47:39.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:39 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:39.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:39 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:39.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:39 smithi149 ceph-mon[33409]: from='osd.2 ' entity='osd.2' 2024-05-03T17:47:39.939 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:47:40.344 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:47:40.344 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:47:40.344 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:40.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:40.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:47:40.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:40.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:40.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: osd.2 [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110] boot 2024-05-03T17:47:40.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T17:47:40.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:40 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:40.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:47:40.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:47:40.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:40.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:40.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:47:40.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:40.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:40.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: osd.2 [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110] boot 2024-05-03T17:47:40.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T17:47:40.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:40 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:40.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:47:40.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:47:40.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T17:47:40.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:40.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:47:40.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:40.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:47:40.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:40.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: osd.2 [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110] boot 2024-05-03T17:47:40.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T17:47:40.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:40 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T17:47:41.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.536+0000 7fc4839af700 1 -- 172.21.15.122:0/2216616494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47c103600 msgr2=0x7fc47c103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:47:41.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.536+0000 7fc4839af700 1 --2- 172.21.15.122:0/2216616494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47c103600 0x7fc47c103a60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0099c0 tx=0x7fc46c009cd0 comp rx=0 tx=0).stop 2024-05-03T17:47:41.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.536+0000 7fc4839af700 1 -- 172.21.15.122:0/2216616494 shutdown_connections 2024-05-03T17:47:41.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.536+0000 7fc4839af700 1 --2- 172.21.15.122:0/2216616494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103fa0 0x7fc47c10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:41.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.536+0000 7fc4839af700 1 --2- 172.21.15.122:0/2216616494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c102400 0x7fc47c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:41.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.536+0000 7fc4839af700 1 --2- 172.21.15.122:0/2216616494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47c103600 0x7fc47c103a60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:41.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.536+0000 7fc4839af700 1 -- 172.21.15.122:0/2216616494 >> 172.21.15.122:0/2216616494 conn(0x7fc47c0fd9a0 msgr2=0x7fc47c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:47:41.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.536+0000 7fc4839af700 1 -- 172.21.15.122:0/2216616494 shutdown_connections 2024-05-03T17:47:41.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.536+0000 7fc4839af700 1 -- 172.21.15.122:0/2216616494 wait complete. 2024-05-03T17:47:41.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.536+0000 7fc4839af700 1 Processor -- start 2024-05-03T17:47:41.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.536+0000 7fc4839af700 1 -- start start 2024-05-03T17:47:41.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc4839af700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c102400 0x7fc47c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:47:41.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc4839af700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47c103600 0x7fc47c19e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:47:41.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc4839af700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103fa0 0x7fc47c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:47:41.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc4839af700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c10de80 con 0x7fc47c103600 2024-05-03T17:47:41.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc4839af700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c10dd00 con 0x7fc47c102400 2024-05-03T17:47:41.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc4839af700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c10e000 con 0x7fc47c103fa0 2024-05-03T17:47:41.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc480f4a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47c103600 0x7fc47c19e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:47:41.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc480f4a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47c103600 0x7fc47c19e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57852/0 (socket says 172.21.15.122:57852) 2024-05-03T17:47:41.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc480f4a700 1 -- 172.21.15.122:0/3860349053 learned_addr learned my addr 172.21.15.122:0/3860349053 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:47:41.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc481f4c700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103fa0 0x7fc47c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:47:41.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc48174b700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c102400 0x7fc47c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:47:41.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc480f4a700 1 -- 172.21.15.122:0/3860349053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103fa0 msgr2=0x7fc47c1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:47:41.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc480f4a700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103fa0 0x7fc47c1a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:41.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc480f4a700 1 -- 172.21.15.122:0/3860349053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c102400 msgr2=0x7fc47c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:47:41.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc480f4a700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c102400 0x7fc47c06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:47:41.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc480f4a700 1 -- 172.21.15.122:0/3860349053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47c1a12e0 con 0x7fc47c103600 2024-05-03T17:47:41.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc480f4a700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47c103600 0x7fc47c19e380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00b580 tx=0x7fc46c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:47:41.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.537+0000 7fc4727fc700 1 -- 172.21.15.122:0/3860349053 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c027070 con 0x7fc47c103600 2024-05-03T17:47:41.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.538+0000 7fc4727fc700 1 -- 172.21.15.122:0/3860349053 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc46c005960 con 0x7fc47c103600 2024-05-03T17:47:41.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.538+0000 7fc4839af700 1 -- 172.21.15.122:0/3860349053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47c1a1570 con 0x7fc47c103600 2024-05-03T17:47:41.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.538+0000 7fc4727fc700 1 -- 172.21.15.122:0/3860349053 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c022c10 con 0x7fc47c103600 2024-05-03T17:47:41.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.538+0000 7fc4839af700 1 -- 172.21.15.122:0/3860349053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47c1a1a50 con 0x7fc47c103600 2024-05-03T17:47:41.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.538+0000 7fc4839af700 1 -- 172.21.15.122:0/3860349053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc47c06a190 con 0x7fc47c103600 2024-05-03T17:47:41.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.540+0000 7fc4727fc700 1 -- 172.21.15.122:0/3860349053 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fc46c02a070 con 0x7fc47c103600 2024-05-03T17:47:41.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.540+0000 7fc4727fc700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4680b27f0 0x7fc4680b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:47:41.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.541+0000 7fc48174b700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4680b27f0 0x7fc4680b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:47:41.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.541+0000 7fc4727fc700 1 -- 172.21.15.122:0/3860349053 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7fc46c0e1d50 con 0x7fc47c103600 2024-05-03T17:47:41.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.541+0000 7fc48174b700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4680b27f0 0x7fc4680b4cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc47c103460 tx=0x7fc47800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:47:41.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.544+0000 7fc4727fc700 1 -- 172.21.15.122:0/3860349053 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc46c013070 con 0x7fc47c103600 2024-05-03T17:47:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:41 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi192 2024-05-03T17:47:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:41 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi192 to 17088M 2024-05-03T17:47:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:41 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T17:47:41.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:41 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi192 2024-05-03T17:47:41.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:41 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi192 to 17088M 2024-05-03T17:47:41.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T17:47:41.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:41.757+0000 7fc4839af700 1 -- 172.21.15.122:0/3860349053 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc47c0638e0 con 0x7fc4680b27f0 2024-05-03T17:47:41.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:41 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi192 2024-05-03T17:47:41.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:41 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi192 to 17088M 2024-05-03T17:47:41.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:41 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T17:47:42.575 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 ceph-mon[29744]: pgmap v79: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:42.575 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T17:47:42.575 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 ceph-mon[29744]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T17:47:42.575 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T17:47:42.575 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:47:42.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:47:42.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:42.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46688]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T17:47:42.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46688]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:42.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46688]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:42.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46688]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:42.576 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46676]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T17:47:42.576 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46676]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:42.576 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46676]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:42.576 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46676]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:42.576 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46679]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T17:47:42.576 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46679]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:42.577 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46679]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:42.577 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46679]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:42.597 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42842]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T17:47:42.598 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42842]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:42.598 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42842]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:42.598 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42842]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:42.598 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42845]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T17:47:42.598 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42845]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:42.598 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42845]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:42.598 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42845]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:42.598 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 ceph-mon[33753]: pgmap v79: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:42.598 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T17:47:42.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 ceph-mon[33753]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T17:47:42.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T17:47:42.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:47:42.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:47:42.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:42.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42848]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T17:47:42.635 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 ceph-mon[33409]: pgmap v79: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:42.635 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T17:47:42.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 ceph-mon[33409]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T17:47:42.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T17:47:42.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:47:42.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:47:42.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:42.636 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42141]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T17:47:42.636 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42141]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:42.636 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42141]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:42.636 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42141]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:42.637 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42144]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T17:47:42.637 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42144]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:42.637 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42144]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:42.637 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42144]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:42.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42848]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:42.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42848]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:42.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42848]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:42.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42851]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T17:47:42.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42851]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:42.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42851]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:42.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:42 smithi192 sudo[42851]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:42.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46691]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T17:47:42.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46691]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:42.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46691]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:42.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:42 smithi122 sudo[46691]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:43.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42147]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T17:47:43.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42147]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:43.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42147]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:43.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42147]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:43.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42150]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T17:47:43.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42150]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T17:47:43.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42150]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T17:47:43.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:42 smithi149 sudo[42150]: pam_unix(sudo:session): session closed for user root 2024-05-03T17:47:43.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:47:43.495+0000 7fc4727fc700 1 -- 172.21.15.122:0/3860349053 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fc46c0a6b50 con 0x7fc47c103600 2024-05-03T17:47:43.566 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:47:43.566 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T17:47:43.566 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T17:47:43.566 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:43.566 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:43.566 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:43.566 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T17:47:43.566 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:47:43.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:47:43.568 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:43 smithi122 ceph-mon[29744]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T17:47:43.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:47:43.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T17:47:43.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T17:47:43.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:47:43.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:47:43.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:47:43.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:43 smithi192 ceph-mon[33753]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:47:43.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T17:47:43.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:47:43.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:47:43.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:47:43.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T17:47:43.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T17:47:43.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T17:47:43.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T17:47:43.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T17:47:43.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:43 smithi149 ceph-mon[33409]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T17:47:44.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:44 smithi192 ceph-mon[33753]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:44.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:44 smithi192 ceph-mon[33753]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T17:47:44.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:44 smithi192 ceph-mon[33753]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-05-03T17:47:44.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:44 smithi122 ceph-mon[29744]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:44.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:44 smithi122 ceph-mon[29744]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T17:47:44.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:44 smithi122 ceph-mon[29744]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-05-03T17:47:44.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:44 smithi149 ceph-mon[33409]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:44.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:44 smithi149 ceph-mon[33409]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T17:47:44.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:44 smithi149 ceph-mon[33409]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-05-03T17:47:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:45 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/1633760431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9973c0c1-ae73-497a-845e-664a20666df6"}]: dispatch 2024-05-03T17:47:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:45 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/1633760431' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9973c0c1-ae73-497a-845e-664a20666df6"}]': finished 2024-05-03T17:47:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:45 smithi192 ceph-mon[33753]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T17:47:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:47:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:45 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/868241230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:47:45.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:45 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1633760431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9973c0c1-ae73-497a-845e-664a20666df6"}]: dispatch 2024-05-03T17:47:45.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:45 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1633760431' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9973c0c1-ae73-497a-845e-664a20666df6"}]': finished 2024-05-03T17:47:45.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:45 smithi122 ceph-mon[29744]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T17:47:45.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:47:45.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:45 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/868241230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:47:45.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:45 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/1633760431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9973c0c1-ae73-497a-845e-664a20666df6"}]: dispatch 2024-05-03T17:47:45.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:45 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/1633760431' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9973c0c1-ae73-497a-845e-664a20666df6"}]': finished 2024-05-03T17:47:45.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:45 smithi149 ceph-mon[33409]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T17:47:45.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:47:45.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:45 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/868241230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:47:46.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:46 smithi122 ceph-mon[29744]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:46.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:46 smithi149 ceph-mon[33409]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:46.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:46 smithi192 ceph-mon[33753]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:48.699 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:48 smithi122 ceph-mon[29744]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:48.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:48 smithi149 ceph-mon[33409]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:48.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:48 smithi192 ceph-mon[33753]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:50.532 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:50 smithi122 ceph-mon[29744]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:50.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:50 smithi149 ceph-mon[33409]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:50.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:50 smithi192 ceph-mon[33753]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:52.527 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:52 smithi122 ceph-mon[29744]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:52.527 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T17:47:52.527 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:52.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:52 smithi149 ceph-mon[33409]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:52.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T17:47:52.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:52.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:52 smithi192 ceph-mon[33753]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:52.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T17:47:52.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:47:53.651 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:53 smithi122 ceph-mon[29744]: Deploying daemon osd.3 on smithi122 2024-05-03T17:47:53.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:53 smithi149 ceph-mon[33409]: Deploying daemon osd.3 on smithi122 2024-05-03T17:47:53.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:53 smithi192 ceph-mon[33753]: Deploying daemon osd.3 on smithi122 2024-05-03T17:47:54.703 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:54 smithi122 ceph-mon[29744]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:54.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:54 smithi149 ceph-mon[33409]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:54.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:54 smithi192 ceph-mon[33753]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:56.706 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:56 smithi122 ceph-mon[29744]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:56.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:56 smithi149 ceph-mon[33409]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:56.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:56 smithi192 ceph-mon[33753]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:57.616 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:57.616 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:57.616 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:57.767 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:57.767 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:57.768 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:57.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:47:57.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:57.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:47:58.706 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:47:58 smithi122 ceph-mon[29744]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:58.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:47:58 smithi149 ceph-mon[33409]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:47:58.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:47:58 smithi192 ceph-mon[33753]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:48:00.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.217+0000 7fc4727fc700 1 -- 172.21.15.122:0/3860349053 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc47c0638e0 con 0x7fc4680b27f0 2024-05-03T17:48:00.213 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 3 on host 'smithi122' 2024-05-03T17:48:00.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 -- 172.21.15.122:0/3860349053 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4680b27f0 msgr2=0x7fc4680b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:00.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4680b27f0 0x7fc4680b4cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc47c103460 tx=0x7fc47800a380 comp rx=0 tx=0).stop 2024-05-03T17:48:00.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 -- 172.21.15.122:0/3860349053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47c103600 msgr2=0x7fc47c19e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:00.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47c103600 0x7fc47c19e380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00b580 tx=0x7fc46c009580 comp rx=0 tx=0).stop 2024-05-03T17:48:00.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 -- 172.21.15.122:0/3860349053 shutdown_connections 2024-05-03T17:48:00.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4680b27f0 0x7fc4680b4cb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:00.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103fa0 0x7fc47c1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:00.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c102400 0x7fc47c06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:00.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 --2- 172.21.15.122:0/3860349053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47c103600 0x7fc47c19e380 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:00.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 -- 172.21.15.122:0/3860349053 >> 172.21.15.122:0/3860349053 conn(0x7fc47c0fd9a0 msgr2=0x7fc47c106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:48:00.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 -- 172.21.15.122:0/3860349053 shutdown_connections 2024-05-03T17:48:00.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:00.219+0000 7fc4839af700 1 -- 172.21.15.122:0/3860349053 wait complete. 2024-05-03T17:48:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:00 smithi122 ceph-mon[29744]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:48:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:00 smithi149 ceph-mon[33409]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:48:00.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:00 smithi192 ceph-mon[33753]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:48:00.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:00.868 DEBUG:teuthology.orchestra.run.smithi122:osd.3> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.3.service 2024-05-03T17:48:00.870 INFO:tasks.cephadm:Deploying osd.4 on smithi149 with /dev/vg_nvme/lv_3... 2024-05-03T17:48:00.870 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T17:48:01.119 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T17:48:01.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:01 smithi149 ceph-mon[33409]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T17:48:01.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:01 smithi122 ceph-mon[29744]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T17:48:01.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:01 smithi192 ceph-mon[33753]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T17:48:02.602 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:02 smithi149 ceph-mon[33409]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:48:02.602 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:02 smithi149 ceph-mon[33409]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T17:48:02.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:02 smithi149 ceph-mon[33409]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T17:48:02.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:02 smithi149 ceph-mon[33409]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:48:02.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:02 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:02.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:02 smithi122 ceph-mon[29744]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:48:02.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:02 smithi122 ceph-mon[29744]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T17:48:02.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:02 smithi122 ceph-mon[29744]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T17:48:02.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:02 smithi122 ceph-mon[29744]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:48:02.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:02 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:02.711 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 17:48:02 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3[48823]: 2024-05-03T17:48:02.468+0000 7ff6ef26c700 -1 osd.3 0 waiting for initial osdmap 2024-05-03T17:48:02.711 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 17:48:02 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3[48823]: 2024-05-03T17:48:02.474+0000 7ff6eb05f700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:48:02.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:02 smithi192 ceph-mon[33753]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:48:02.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:02 smithi192 ceph-mon[33753]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T17:48:02.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:02 smithi192 ceph-mon[33753]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T17:48:02.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:02 smithi192 ceph-mon[33753]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:48:02.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:02 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:03.204 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T17:48:03.205 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T17:48:03.205 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-03T17:48:03.205 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-03T17:48:03.205 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013164 s, 797 MB/s 2024-05-03T17:48:03.205 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-03T17:48:03.629 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:03 smithi122 ceph-mon[29744]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:48:03.630 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:03 smithi122 ceph-mon[29744]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T17:48:03.630 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:03.630 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:03 smithi149 ceph-mon[33409]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:48:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:03 smithi149 ceph-mon[33409]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T17:48:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:03.863 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-05-03T17:48:03.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:03 smithi192 ceph-mon[33753]: from='osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:48:03.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:03 smithi192 ceph-mon[33753]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T17:48:03.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:03.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:04.107 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T17:48:04.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:48:04.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:48:04.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:48:04.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:04.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764] boot 2024-05-03T17:48:04.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T17:48:04.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:04.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T17:48:04.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:04.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:04.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:04.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:04.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:04.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:04.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:04.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:48:04.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764] boot 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:04.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:04.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:04.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:48:04.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:48:04.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T17:48:04.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:04.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: osd.3 [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764] boot 2024-05-03T17:48:04.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T17:48:04.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T17:48:04.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T17:48:04.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:04.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:04.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:04.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:04.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:04.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:04.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:04.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:04.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:05.495 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:05 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi122 2024-05-03T17:48:05.496 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:05 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi122 to 8544M 2024-05-03T17:48:05.496 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:05 smithi149 ceph-mon[33409]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T17:48:05.770 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.773+0000 7f02ab5bf700 1 -- 172.21.15.149:0/2912572922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a4103610 msgr2=0x7f02a4103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:05.771 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.773+0000 7f02ab5bf700 1 --2- 172.21.15.149:0/2912572922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a4103610 0x7f02a4103a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02940099c0 tx=0x7f0294009cd0 comp rx=0 tx=0).stop 2024-05-03T17:48:05.771 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.773+0000 7f02ab5bf700 1 -- 172.21.15.149:0/2912572922 shutdown_connections 2024-05-03T17:48:05.771 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.773+0000 7f02ab5bf700 1 --2- 172.21.15.149:0/2912572922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a4103fb0 0x7f02a410a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:05.771 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.773+0000 7f02ab5bf700 1 --2- 172.21.15.149:0/2912572922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02a4102410 0x7f02a4102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:05.771 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.773+0000 7f02ab5bf700 1 --2- 172.21.15.149:0/2912572922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a4103610 0x7f02a4103a70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:05.771 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.773+0000 7f02ab5bf700 1 -- 172.21.15.149:0/2912572922 >> 172.21.15.149:0/2912572922 conn(0x7f02a40fd9d0 msgr2=0x7f02a40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:48:05.771 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.773+0000 7f02ab5bf700 1 -- 172.21.15.149:0/2912572922 shutdown_connections 2024-05-03T17:48:05.771 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.773+0000 7f02ab5bf700 1 -- 172.21.15.149:0/2912572922 wait complete. 2024-05-03T17:48:05.771 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.773+0000 7f02ab5bf700 1 Processor -- start 2024-05-03T17:48:05.771 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.773+0000 7f02ab5bf700 1 -- start start 2024-05-03T17:48:05.771 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02ab5bf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a4102410 0x7f02a41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02ab5bf700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a4103610 0x7f02a4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02ab5bf700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02a4103fb0 0x7f02a419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02ab5bf700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f02a410de90 con 0x7f02a4103fb0 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02ab5bf700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02a410dd10 con 0x7f02a4103610 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02ab5bf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f02a410e010 con 0x7f02a4102410 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a8b5a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a4103610 0x7f02a4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a8b5a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a4103610 0x7f02a4198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:43522/0 (socket says 172.21.15.149:43522) 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a8b5a700 1 -- 172.21.15.149:0/1359633620 learned_addr learned my addr 172.21.15.149:0/1359633620 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a935b700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a4102410 0x7f02a41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a9b5c700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02a4103fb0 0x7f02a419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a8b5a700 1 -- 172.21.15.149:0/1359633620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a4102410 msgr2=0x7f02a41985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a8b5a700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a4102410 0x7f02a41985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:05.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a8b5a700 1 -- 172.21.15.149:0/1359633620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02a4103fb0 msgr2=0x7f02a419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:05.773 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a8b5a700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02a4103fb0 0x7f02a419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:05.773 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a8b5a700 1 -- 172.21.15.149:0/1359633620 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a41a0220 con 0x7f02a4103610 2024-05-03T17:48:05.773 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a935b700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a4102410 0x7f02a41985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:48:05.773 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f02a8b5a700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a4103610 0x7f02a4198b00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0294000c00 tx=0x7f029401ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:48:05.773 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.774+0000 7f029a7fc700 1 -- 172.21.15.149:0/1359633620 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029402a070 con 0x7f02a4103610 2024-05-03T17:48:05.773 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.775+0000 7f029a7fc700 1 -- 172.21.15.149:0/1359633620 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f029400f910 con 0x7f02a4103610 2024-05-03T17:48:05.773 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.775+0000 7f02ab5bf700 1 -- 172.21.15.149:0/1359633620 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a406bbf0 con 0x7f02a4103610 2024-05-03T17:48:05.773 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.775+0000 7f029a7fc700 1 -- 172.21.15.149:0/1359633620 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294009e60 con 0x7f02a4103610 2024-05-03T17:48:05.773 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.775+0000 7f02ab5bf700 1 -- 172.21.15.149:0/1359633620 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a406c0d0 con 0x7f02a4103610 2024-05-03T17:48:05.774 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.776+0000 7f02ab5bf700 1 -- 172.21.15.149:0/1359633620 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a406a190 con 0x7f02a4103610 2024-05-03T17:48:05.774 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.777+0000 7f029a7fc700 1 -- 172.21.15.149:0/1359633620 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f029402d070 con 0x7f02a4103610 2024-05-03T17:48:05.775 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.778+0000 7f029a7fc700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f02900b2880 0x7f02900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:05.775 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.778+0000 7f029a7fc700 1 -- 172.21.15.149:0/1359633620 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f02940e4250 con 0x7f02a4103610 2024-05-03T17:48:05.775 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.778+0000 7f02a935b700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f02900b2880 0x7f02900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:05.775 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.779+0000 7f02a935b700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f02900b2880 0x7f02900b4d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02a4103470 tx=0x7f02a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:48:05.777 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.782+0000 7f029a7fc700 1 -- 172.21.15.149:0/1359633620 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0294034080 con 0x7f02a4103610 2024-05-03T17:48:05.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:05 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi122 2024-05-03T17:48:05.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:05 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi122 to 8544M 2024-05-03T17:48:05.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:05 smithi192 ceph-mon[33753]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T17:48:05.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:05 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi122 2024-05-03T17:48:05.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:05 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi122 to 8544M 2024-05-03T17:48:05.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:05 smithi122 ceph-mon[29744]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T17:48:05.987 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:05.991+0000 7f02ab5bf700 1 -- 172.21.15.149:0/1359633620 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f02a406c3f0 con 0x7f02900b2880 2024-05-03T17:48:06.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:06 smithi149 ceph-mon[33409]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:06.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:48:06.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:48:06.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:06.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:06 smithi192 ceph-mon[33753]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:06.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:48:06.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:48:06.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:06.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:06 smithi122 ceph-mon[29744]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:06.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:48:06.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:48:06.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:07.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:07 smithi149 ceph-mon[33409]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:48:07.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:07 smithi192 ceph-mon[33753]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:48:07.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:07 smithi122 ceph-mon[29744]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:48:08.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:08 smithi149 ceph-mon[33409]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-05-03T17:48:08.861 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:08 smithi122 ceph-mon[29744]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-05-03T17:48:08.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:08 smithi192 ceph-mon[33753]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-05-03T17:48:09.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:09 smithi149 ceph-mon[33409]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T17:48:09.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:09 smithi149 ceph-mon[33409]: from='client.? 172.21.15.149:0/3605476334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da71dc18-94b9-4f7d-a061-7cc7b77db282"}]: dispatch 2024-05-03T17:48:09.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:09 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da71dc18-94b9-4f7d-a061-7cc7b77db282"}]: dispatch 2024-05-03T17:48:09.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:09 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da71dc18-94b9-4f7d-a061-7cc7b77db282"}]': finished 2024-05-03T17:48:09.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:09 smithi149 ceph-mon[33409]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T17:48:09.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:09 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:09.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:09 smithi149 ceph-mon[33409]: from='client.? 172.21.15.149:0/1473044388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:48:09.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:09 smithi192 ceph-mon[33753]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T17:48:09.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:09 smithi192 ceph-mon[33753]: from='client.? 172.21.15.149:0/3605476334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da71dc18-94b9-4f7d-a061-7cc7b77db282"}]: dispatch 2024-05-03T17:48:09.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:09 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da71dc18-94b9-4f7d-a061-7cc7b77db282"}]: dispatch 2024-05-03T17:48:09.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:09 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da71dc18-94b9-4f7d-a061-7cc7b77db282"}]': finished 2024-05-03T17:48:09.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:09 smithi192 ceph-mon[33753]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T17:48:09.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:09 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:09.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:09 smithi192 ceph-mon[33753]: from='client.? 172.21.15.149:0/1473044388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:48:09.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:09 smithi122 ceph-mon[29744]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T17:48:09.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:09 smithi122 ceph-mon[29744]: from='client.? 172.21.15.149:0/3605476334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da71dc18-94b9-4f7d-a061-7cc7b77db282"}]: dispatch 2024-05-03T17:48:09.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:09 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da71dc18-94b9-4f7d-a061-7cc7b77db282"}]: dispatch 2024-05-03T17:48:09.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:09 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da71dc18-94b9-4f7d-a061-7cc7b77db282"}]': finished 2024-05-03T17:48:09.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:09 smithi122 ceph-mon[29744]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T17:48:09.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:09 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:09.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:09 smithi122 ceph-mon[29744]: from='client.? 172.21.15.149:0/1473044388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:48:11.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:11 smithi149 ceph-mon[33409]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:48:11.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:11 smithi192 ceph-mon[33753]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:48:11.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:11 smithi122 ceph-mon[29744]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:48:13.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:13 smithi149 ceph-mon[33409]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:48:13.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:13 smithi192 ceph-mon[33753]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:48:13.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:13 smithi122 ceph-mon[29744]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:48:15.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:15 smithi149 ceph-mon[33409]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:48:15.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:15 smithi192 ceph-mon[33753]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:48:15.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:15 smithi122 ceph-mon[29744]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:48:16.819 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T17:48:16.819 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:16.819 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:16 smithi149 ceph-mon[33409]: Deploying daemon osd.4 on smithi149 2024-05-03T17:48:16.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T17:48:16.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:16.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:16 smithi192 ceph-mon[33753]: Deploying daemon osd.4 on smithi149 2024-05-03T17:48:16.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T17:48:16.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:16.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:16 smithi122 ceph-mon[29744]: Deploying daemon osd.4 on smithi149 2024-05-03T17:48:17.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:17 smithi149 ceph-mon[33409]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:17.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:17 smithi192 ceph-mon[33753]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:17.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:17 smithi122 ceph-mon[29744]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:19.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:19 smithi149 ceph-mon[33409]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:19.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:19 smithi192 ceph-mon[33753]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:19.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:19 smithi122 ceph-mon[29744]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:21.584 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:21 smithi149 ceph-mon[33409]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:21.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:21 smithi192 ceph-mon[33753]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:21.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:21 smithi122 ceph-mon[29744]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:23.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:22 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:23.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:22 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:23.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:22 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:23.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:22 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:23.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:22 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:23.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:22 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:23.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:22 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:23.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:22 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:23.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:22 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:24.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:23 smithi149 ceph-mon[33409]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:24.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:23 smithi192 ceph-mon[33753]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:24.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:23 smithi122 ceph-mon[29744]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:25.403 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.407+0000 7f029a7fc700 1 -- 172.21.15.149:0/1359633620 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f02a406c3f0 con 0x7f02900b2880 2024-05-03T17:48:25.404 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 4 on host 'smithi149' 2024-05-03T17:48:25.406 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.409+0000 7f02ab5bf700 1 -- 172.21.15.149:0/1359633620 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f02900b2880 msgr2=0x7f02900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:25.406 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.409+0000 7f02ab5bf700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f02900b2880 0x7f02900b4d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02a4103470 tx=0x7f02a0009260 comp rx=0 tx=0).stop 2024-05-03T17:48:25.406 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.410+0000 7f02ab5bf700 1 -- 172.21.15.149:0/1359633620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a4103610 msgr2=0x7f02a4198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:25.406 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.410+0000 7f02ab5bf700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a4103610 0x7f02a4198b00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0294000c00 tx=0x7f029401ac60 comp rx=0 tx=0).stop 2024-05-03T17:48:25.407 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.410+0000 7f02ab5bf700 1 -- 172.21.15.149:0/1359633620 shutdown_connections 2024-05-03T17:48:25.407 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.410+0000 7f02ab5bf700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f02900b2880 0x7f02900b4d40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:25.407 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.410+0000 7f02ab5bf700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02a4103fb0 0x7f02a419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:25.407 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.410+0000 7f02ab5bf700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a4102410 0x7f02a41985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:25.407 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.410+0000 7f02ab5bf700 1 --2- 172.21.15.149:0/1359633620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02a4103610 0x7f02a4198b00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:25.407 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.410+0000 7f02ab5bf700 1 -- 172.21.15.149:0/1359633620 >> 172.21.15.149:0/1359633620 conn(0x7f02a40fd9d0 msgr2=0x7f02a4106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:48:25.407 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.410+0000 7f02ab5bf700 1 -- 172.21.15.149:0/1359633620 shutdown_connections 2024-05-03T17:48:25.407 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:48:25.410+0000 7f02ab5bf700 1 -- 172.21.15.149:0/1359633620 wait complete. 2024-05-03T17:48:25.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:25.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:25.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.618 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:25.618 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:25.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:25.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:25.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:25.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:25.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:25.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:25.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:26.024 DEBUG:teuthology.orchestra.run.smithi149:osd.4> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.4.service 2024-05-03T17:48:26.026 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_3... 2024-05-03T17:48:26.026 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T17:48:26.272 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:48:26.656 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:26 smithi192 ceph-mon[33753]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:26.657 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:26 smithi192 ceph-mon[33753]: from='osd.4 [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T17:48:26.657 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:26 smithi192 ceph-mon[33753]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T17:48:26.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:26 smithi149 ceph-mon[33409]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:26.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:26 smithi149 ceph-mon[33409]: from='osd.4 [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T17:48:26.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:26 smithi149 ceph-mon[33409]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T17:48:26.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:26 smithi122 ceph-mon[29744]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:26.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:26 smithi122 ceph-mon[29744]: from='osd.4 [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T17:48:26.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:26 smithi122 ceph-mon[29744]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T17:48:27.542 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:27 smithi192 ceph-mon[33753]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T17:48:27.542 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:27 smithi192 ceph-mon[33753]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T17:48:27.542 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:27 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:27.542 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:27 smithi192 ceph-mon[33753]: from='osd.4 [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:48:27.542 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:27 smithi192 ceph-mon[33753]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:48:27.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:27 smithi149 ceph-mon[33409]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T17:48:27.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:27 smithi149 ceph-mon[33409]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T17:48:27.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:27 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:27.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:27 smithi149 ceph-mon[33409]: from='osd.4 [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:48:27.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:27 smithi149 ceph-mon[33409]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:48:27.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:27 smithi122 ceph-mon[29744]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T17:48:27.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:27 smithi122 ceph-mon[29744]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T17:48:27.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:27 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:27.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:27 smithi122 ceph-mon[29744]: from='osd.4 [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:48:27.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:27 smithi122 ceph-mon[29744]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:48:28.116 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T17:48:28.116 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T17:48:28.116 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-05-03T17:48:28.116 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-05-03T17:48:28.116 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128784 s, 814 MB/s 2024-05-03T17:48:28.116 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-05-03T17:48:28.177 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 17:48:27 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4[44962]: 2024-05-03T17:48:27.916+0000 7f9f4595a700 -1 osd.4 0 waiting for initial osdmap 2024-05-03T17:48:28.177 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 17:48:27 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4[44962]: 2024-05-03T17:48:27.922+0000 7f9f3d745700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:48:28.685 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:28 smithi192 ceph-mon[33753]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:28.685 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:28 smithi192 ceph-mon[33753]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:48:28.685 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:28 smithi192 ceph-mon[33753]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T17:48:28.685 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:28 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:28.685 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:28 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:28.685 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:28 smithi192 ceph-mon[33753]: from='osd.4 ' entity='osd.4' 2024-05-03T17:48:28.712 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2024-05-03T17:48:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:28 smithi149 ceph-mon[33409]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:28 smithi149 ceph-mon[33409]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:48:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:28 smithi149 ceph-mon[33409]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T17:48:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:28 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:28 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:28 smithi149 ceph-mon[33409]: from='osd.4 ' entity='osd.4' 2024-05-03T17:48:28.955 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:48:28.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:28 smithi122 ceph-mon[29744]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:28.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:28 smithi122 ceph-mon[29744]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:48:28.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:28 smithi122 ceph-mon[29744]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T17:48:28.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:28 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:28.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:28 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:28.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:28 smithi122 ceph-mon[29744]: from='osd.4 ' entity='osd.4' 2024-05-03T17:48:29.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:48:29.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:48:29.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:29.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:29.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:29.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:29.751 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:29.751 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.751 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: osd.4 [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532] boot 2024-05-03T17:48:29.751 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T17:48:29.751 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:29.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:48:29.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: osd.4 [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532] boot 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T17:48:29.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:29.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:48:29.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:48:29.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:29.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:29.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:29.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:29.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:29.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:29.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: osd.4 [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532] boot 2024-05-03T17:48:29.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T17:48:29.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T17:48:30.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.502+0000 7ff51516e700 1 -- 172.21.15.192:0/2486718387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff510104730 msgr2=0x7ff510104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:30.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.502+0000 7ff51516e700 1 --2- 172.21.15.192:0/2486718387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff510104730 0x7ff510104b90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff504009990 tx=0x7ff504009ca0 comp rx=0 tx=0).stop 2024-05-03T17:48:30.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.502+0000 7ff51516e700 1 -- 172.21.15.192:0/2486718387 shutdown_connections 2024-05-03T17:48:30.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.502+0000 7ff51516e700 1 --2- 172.21.15.192:0/2486718387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5101050d0 0x7ff51010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:30.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.502+0000 7ff51516e700 1 --2- 172.21.15.192:0/2486718387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510103530 0x7ff510103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:30.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.502+0000 7ff51516e700 1 --2- 172.21.15.192:0/2486718387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff510104730 0x7ff510104b90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:30.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.502+0000 7ff51516e700 1 -- 172.21.15.192:0/2486718387 >> 172.21.15.192:0/2486718387 conn(0x7ff5100feaf0 msgr2=0x7ff510100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:48:30.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.502+0000 7ff51516e700 1 -- 172.21.15.192:0/2486718387 shutdown_connections 2024-05-03T17:48:30.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.502+0000 7ff51516e700 1 -- 172.21.15.192:0/2486718387 wait complete. 2024-05-03T17:48:30.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff51516e700 1 Processor -- start 2024-05-03T17:48:30.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff51516e700 1 -- start start 2024-05-03T17:48:30.500 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff51516e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510103530 0x7ff510199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff51516e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff510104730 0x7ff510199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff51516e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5101050d0 0x7ff5101a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff51516e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff51010efc0 con 0x7ff5101050d0 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff51516e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff51010ee40 con 0x7ff510103530 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff51516e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff51010f140 con 0x7ff510104730 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff50e59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff510104730 0x7ff510199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff50e59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff510104730 0x7ff510199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:46702/0 (socket says 172.21.15.192:46702) 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff50e59c700 1 -- 172.21.15.192:0/1467522679 learned_addr learned my addr 172.21.15.192:0/1467522679 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff50ed9d700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510103530 0x7ff510199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff50f59e700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5101050d0 0x7ff5101a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff50e59c700 1 -- 172.21.15.192:0/1467522679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510103530 msgr2=0x7ff510199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff50e59c700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510103530 0x7ff510199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff50e59c700 1 -- 172.21.15.192:0/1467522679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5101050d0 msgr2=0x7ff5101a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff50e59c700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5101050d0 0x7ff5101a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.503+0000 7ff50e59c700 1 -- 172.21.15.192:0/1467522679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5101a12a0 con 0x7ff510104730 2024-05-03T17:48:30.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.504+0000 7ff50e59c700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff510104730 0x7ff510199b80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff5040036f0 tx=0x7ff5040047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:48:30.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.504+0000 7ff4f7fff700 1 -- 172.21.15.192:0/1467522679 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504027070 con 0x7ff510104730 2024-05-03T17:48:30.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.504+0000 7ff4f7fff700 1 -- 172.21.15.192:0/1467522679 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff504004000 con 0x7ff510104730 2024-05-03T17:48:30.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.504+0000 7ff51516e700 1 -- 172.21.15.192:0/1467522679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5101a1530 con 0x7ff510104730 2024-05-03T17:48:30.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.504+0000 7ff4f7fff700 1 -- 172.21.15.192:0/1467522679 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504022c60 con 0x7ff510104730 2024-05-03T17:48:30.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.504+0000 7ff51516e700 1 -- 172.21.15.192:0/1467522679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5101a1980 con 0x7ff510104730 2024-05-03T17:48:30.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.505+0000 7ff51516e700 1 -- 172.21.15.192:0/1467522679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51006a190 con 0x7ff510104730 2024-05-03T17:48:30.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.507+0000 7ff4f7fff700 1 -- 172.21.15.192:0/1467522679 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff50402a070 con 0x7ff510104730 2024-05-03T17:48:30.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.507+0000 7ff4f7fff700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff4f80b2880 0x7ff4f80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:30.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.507+0000 7ff4f7fff700 1 -- 172.21.15.192:0/1467522679 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7ff5040e2870 con 0x7ff510104730 2024-05-03T17:48:30.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.508+0000 7ff50ed9d700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff4f80b2880 0x7ff4f80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:30.504 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.508+0000 7ff50ed9d700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff4f80b2880 0x7ff4f80b4d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff510104590 tx=0x7ff50000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:48:30.506 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.511+0000 7ff4f7fff700 1 -- 172.21.15.192:0/1467522679 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff504031070 con 0x7ff510104730 2024-05-03T17:48:30.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:30 smithi192 ceph-mon[33753]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:30.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:30 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi149 2024-05-03T17:48:30.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:30 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi149 to 2908M 2024-05-03T17:48:30.715 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:30.720+0000 7ff51516e700 1 -- 172.21.15.192:0/1467522679 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff5100638e0 con 0x7ff4f80b2880 2024-05-03T17:48:30.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:30 smithi149 ceph-mon[33409]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:30.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:30 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi149 2024-05-03T17:48:30.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:30 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi149 to 2908M 2024-05-03T17:48:30.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:30 smithi122 ceph-mon[29744]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-03T17:48:30.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:30 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi149 2024-05-03T17:48:30.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:30 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi149 to 2908M 2024-05-03T17:48:31.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:31 smithi192 ceph-mon[33753]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:31.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:31 smithi192 ceph-mon[33753]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T17:48:31.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:31 smithi192 ceph-mon[33753]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:48:31.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:31 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:48:31.607 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:31 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:48:31.608 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:31 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:31.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:31 smithi149 ceph-mon[33409]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:31.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:31 smithi149 ceph-mon[33409]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T17:48:31.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:31 smithi149 ceph-mon[33409]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:48:31.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:31 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:48:31.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:31 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:48:31.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:31 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:31.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:31 smithi122 ceph-mon[29744]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:31.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:31 smithi122 ceph-mon[29744]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T17:48:31.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:31 smithi122 ceph-mon[29744]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:48:31.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:31 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:48:31.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:31 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:48:31.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:31 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:33.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:33 smithi192 ceph-mon[33753]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:33.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:33 smithi192 ceph-mon[33753]: from='client.? 172.21.15.192:0/2677581394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec"}]: dispatch 2024-05-03T17:48:33.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:33 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec"}]: dispatch 2024-05-03T17:48:33.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:33 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec"}]': finished 2024-05-03T17:48:33.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:33 smithi192 ceph-mon[33753]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T17:48:33.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:33.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:33 smithi149 ceph-mon[33409]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:33.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:33 smithi149 ceph-mon[33409]: from='client.? 172.21.15.192:0/2677581394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec"}]: dispatch 2024-05-03T17:48:33.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:33 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec"}]: dispatch 2024-05-03T17:48:33.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:33 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec"}]': finished 2024-05-03T17:48:33.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:33 smithi149 ceph-mon[33409]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T17:48:33.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:33.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:33 smithi122 ceph-mon[29744]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:33.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:33 smithi122 ceph-mon[29744]: from='client.? 172.21.15.192:0/2677581394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec"}]: dispatch 2024-05-03T17:48:33.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:33 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec"}]: dispatch 2024-05-03T17:48:33.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:33 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec"}]': finished 2024-05-03T17:48:33.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:33 smithi122 ceph-mon[29744]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T17:48:33.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:34.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:34 smithi149 ceph-mon[33409]: from='client.? 172.21.15.192:0/190030929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:48:34.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:34 smithi192 ceph-mon[33753]: from='client.? 172.21.15.192:0/190030929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:48:34.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:34 smithi122 ceph-mon[29744]: from='client.? 172.21.15.192:0/190030929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:48:35.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:35 smithi149 ceph-mon[33409]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:35.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:35 smithi192 ceph-mon[33753]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:35.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:35 smithi122 ceph-mon[29744]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:37.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:37 smithi149 ceph-mon[33409]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:37.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:37 smithi192 ceph-mon[33753]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:37.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:37 smithi122 ceph-mon[29744]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:39.712 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:39 smithi192 ceph-mon[33753]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:39.762 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:39 smithi122 ceph-mon[29744]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:39.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:39 smithi149 ceph-mon[33409]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:41.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:41 smithi149 ceph-mon[33409]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:41.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:41 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T17:48:41.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:41 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:41.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:41 smithi149 ceph-mon[33409]: Deploying daemon osd.5 on smithi192 2024-05-03T17:48:41.840 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:41 smithi192 ceph-mon[33753]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:41.840 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:41 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T17:48:41.840 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:41 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:41.840 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:41 smithi192 ceph-mon[33753]: Deploying daemon osd.5 on smithi192 2024-05-03T17:48:41.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:41 smithi122 ceph-mon[29744]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:41.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T17:48:41.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:41.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:41 smithi122 ceph-mon[29744]: Deploying daemon osd.5 on smithi192 2024-05-03T17:48:43.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:43 smithi149 ceph-mon[33409]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:43.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:43 smithi192 ceph-mon[33753]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:43.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:43 smithi122 ceph-mon[29744]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:45.785 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:45 smithi192 ceph-mon[33753]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:45.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:45 smithi149 ceph-mon[33409]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:45.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:45 smithi122 ceph-mon[29744]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:47.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:47 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:47.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:47 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:47.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:47 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:47.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:47 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:47.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:47 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:47.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:47 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:47.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:47 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:47.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:47 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:47.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:47 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:48.600 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:48 smithi192 ceph-mon[33753]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:48.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:48 smithi122 ceph-mon[29744]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:48.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:48 smithi149 ceph-mon[33409]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:49.781 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.786+0000 7ff4f7fff700 1 -- 172.21.15.192:0/1467522679 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff5100638e0 con 0x7ff4f80b2880 2024-05-03T17:48:49.782 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2024-05-03T17:48:49.784 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 -- 172.21.15.192:0/1467522679 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff4f80b2880 msgr2=0x7ff4f80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:49.784 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff4f80b2880 0x7ff4f80b4d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff510104590 tx=0x7ff50000b410 comp rx=0 tx=0).stop 2024-05-03T17:48:49.784 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 -- 172.21.15.192:0/1467522679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff510104730 msgr2=0x7ff510199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:49.784 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff510104730 0x7ff510199b80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff5040036f0 tx=0x7ff5040047b0 comp rx=0 tx=0).stop 2024-05-03T17:48:49.784 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 -- 172.21.15.192:0/1467522679 shutdown_connections 2024-05-03T17:48:49.784 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff4f80b2880 0x7ff4f80b4d40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:49.784 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5101050d0 0x7ff5101a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:49.784 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510103530 0x7ff510199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:49.784 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 --2- 172.21.15.192:0/1467522679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff510104730 0x7ff510199b80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:49.784 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 -- 172.21.15.192:0/1467522679 >> 172.21.15.192:0/1467522679 conn(0x7ff5100feaf0 msgr2=0x7ff510107ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:48:49.785 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 -- 172.21.15.192:0/1467522679 shutdown_connections 2024-05-03T17:48:49.785 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:48:49.788+0000 7ff51516e700 1 -- 172.21.15.192:0/1467522679 wait complete. 2024-05-03T17:48:50.449 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.5.service 2024-05-03T17:48:50.451 INFO:tasks.cephadm:Deploying osd.6 on smithi122 with /dev/vg_nvme/lv_2... 2024-05-03T17:48:50.451 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T17:48:50.687 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:50 smithi192 ceph-mon[33753]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:50.687 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:50 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.687 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:50 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.688 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:50 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.688 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:50 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.688 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:50 smithi192 ceph-mon[33753]: from='osd.5 [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T17:48:50.688 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:50 smithi192 ceph-mon[33753]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T17:48:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:48:50.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:50 smithi122 ceph-mon[29744]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:50.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:50 smithi122 ceph-mon[29744]: from='osd.5 [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T17:48:50.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:50 smithi122 ceph-mon[29744]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T17:48:50.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:50 smithi149 ceph-mon[33409]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:50.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:50 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:50 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:50 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:50 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:50.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:50 smithi149 ceph-mon[33409]: from='osd.5 [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T17:48:50.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:50 smithi149 ceph-mon[33409]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T17:48:52.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:52 smithi192 ceph-mon[33753]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:52.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:52 smithi192 ceph-mon[33753]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T17:48:52.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:52 smithi192 ceph-mon[33753]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T17:48:52.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:52.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:52 smithi192 ceph-mon[33753]: from='osd.5 [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:48:52.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:52 smithi192 ceph-mon[33753]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:48:52.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:52 smithi122 ceph-mon[29744]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:52.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:52 smithi122 ceph-mon[29744]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T17:48:52.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:52 smithi122 ceph-mon[29744]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T17:48:52.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:52.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:52 smithi122 ceph-mon[29744]: from='osd.5 [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:48:52.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:52 smithi122 ceph-mon[29744]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:48:52.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:52 smithi149 ceph-mon[33409]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:52.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:52 smithi149 ceph-mon[33409]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T17:48:52.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:52 smithi149 ceph-mon[33409]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T17:48:52.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:52.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:52 smithi149 ceph-mon[33409]: from='osd.5 [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:48:52.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:52 smithi149 ceph-mon[33409]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:48:52.649 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T17:48:52.649 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T17:48:52.649 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-05-03T17:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-05-03T17:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126119 s, 831 MB/s 2024-05-03T17:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-05-03T17:48:52.798 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 17:48:52 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T17:48:52.525+0000 7f610081c700 -1 osd.5 0 waiting for initial osdmap 2024-05-03T17:48:52.798 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 17:48:52 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T17:48:52.534+0000 7f60f8607700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:48:53.255 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:53 smithi122 ceph-mon[29744]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:48:53.255 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:53 smithi122 ceph-mon[29744]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T17:48:53.255 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:53 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:53.255 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:53 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:53.255 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:53 smithi122 ceph-mon[29744]: from='osd.5 ' entity='osd.5' 2024-05-03T17:48:53.256 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_2 2024-05-03T17:48:53.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:53 smithi192 ceph-mon[33753]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:48:53.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:53 smithi192 ceph-mon[33753]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T17:48:53.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:53 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:53.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:53 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:53.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:53 smithi192 ceph-mon[33753]: from='osd.5 ' entity='osd.5' 2024-05-03T17:48:53.502 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:48:53.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:53 smithi149 ceph-mon[33409]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:48:53.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:53 smithi149 ceph-mon[33409]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T17:48:53.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:53 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:53.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:53 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:53.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:53 smithi149 ceph-mon[33409]: from='osd.5 ' entity='osd.5' 2024-05-03T17:48:54.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:48:54.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: osd.5 [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485] boot 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T17:48:54.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:54.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:54.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:54.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: osd.5 [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485] boot 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T17:48:54.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:54.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:54.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:54.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:48:54.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:48:54.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: osd.5 [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485] boot 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:54.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T17:48:54.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:48:54.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:54.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:48:54.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:48:55.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.125+0000 7fc5489e3700 1 -- 172.21.15.122:0/417213188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5441068d0 msgr2=0x7fc544108cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:55.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.125+0000 7fc5489e3700 1 --2- 172.21.15.122:0/417213188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5441068d0 0x7fc544108cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009990 tx=0x7fc52c009ca0 comp rx=0 tx=0).stop 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.125+0000 7fc5489e3700 1 -- 172.21.15.122:0/417213188 shutdown_connections 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.125+0000 7fc5489e3700 1 --2- 172.21.15.122:0/417213188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc544109200 0x7fc54410b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.125+0000 7fc5489e3700 1 --2- 172.21.15.122:0/417213188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc544103fa0 0x7fc544106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.125+0000 7fc5489e3700 1 --2- 172.21.15.122:0/417213188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5441068d0 0x7fc544108cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.125+0000 7fc5489e3700 1 -- 172.21.15.122:0/417213188 >> 172.21.15.122:0/417213188 conn(0x7fc5440fd9d0 msgr2=0x7fc5440ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.125+0000 7fc5489e3700 1 -- 172.21.15.122:0/417213188 shutdown_connections 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.125+0000 7fc5489e3700 1 -- 172.21.15.122:0/417213188 wait complete. 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.125+0000 7fc5489e3700 1 Processor -- start 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc5489e3700 1 -- start start 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc5489e3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc544103fa0 0x7fc544199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc5489e3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5441068d0 0x7fc544199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:55.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc5489e3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc544109200 0x7fc5441a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc5489e3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc54410f030 con 0x7fc544103fa0 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc5489e3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc54410eeb0 con 0x7fc5441068d0 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc5489e3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc54410f1b0 con 0x7fc544109200 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc54259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc544103fa0 0x7fc544199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc54259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc544103fa0 0x7fc544199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37098/0 (socket says 172.21.15.122:37098) 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc54259c700 1 -- 172.21.15.122:0/1116450524 learned_addr learned my addr 172.21.15.122:0/1116450524 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc542d9d700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc544109200 0x7fc5441a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc539d9b700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5441068d0 0x7fc544199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc54259c700 1 -- 172.21.15.122:0/1116450524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc544109200 msgr2=0x7fc5441a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc54259c700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc544109200 0x7fc5441a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc54259c700 1 -- 172.21.15.122:0/1116450524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5441068d0 msgr2=0x7fc544199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc54259c700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5441068d0 0x7fc544199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.126+0000 7fc54259c700 1 -- 172.21.15.122:0/1116450524 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5441a1360 con 0x7fc544103fa0 2024-05-03T17:48:55.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.127+0000 7fc54259c700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc544103fa0 0x7fc544199700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc53400d9b0 tx=0x7fc53400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:48:55.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.127+0000 7fc53bfff700 1 -- 172.21.15.122:0/1116450524 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534017070 con 0x7fc544103fa0 2024-05-03T17:48:55.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.127+0000 7fc5489e3700 1 -- 172.21.15.122:0/1116450524 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5441a1650 con 0x7fc544103fa0 2024-05-03T17:48:55.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.127+0000 7fc53bfff700 1 -- 172.21.15.122:0/1116450524 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc534005020 con 0x7fc544103fa0 2024-05-03T17:48:55.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.127+0000 7fc53bfff700 1 -- 172.21.15.122:0/1116450524 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534009b50 con 0x7fc544103fa0 2024-05-03T17:48:55.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.127+0000 7fc5489e3700 1 -- 172.21.15.122:0/1116450524 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5441a1bc0 con 0x7fc544103fa0 2024-05-03T17:48:55.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.128+0000 7fc5489e3700 1 -- 172.21.15.122:0/1116450524 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54406a190 con 0x7fc544103fa0 2024-05-03T17:48:55.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.129+0000 7fc53bfff700 1 -- 172.21.15.122:0/1116450524 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fc5340051d0 con 0x7fc544103fa0 2024-05-03T17:48:55.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.130+0000 7fc53bfff700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc5300b2880 0x7fc5300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:48:55.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.130+0000 7fc539d9b700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc5300b2880 0x7fc5300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:48:55.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.130+0000 7fc53bfff700 1 -- 172.21.15.122:0/1116450524 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7fc5340e16a0 con 0x7fc544103fa0 2024-05-03T17:48:55.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.130+0000 7fc539d9b700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc5300b2880 0x7fc5300b4d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00b5c0 tx=0x7fc52c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:48:55.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.133+0000 7fc53bfff700 1 -- 172.21.15.122:0/1116450524 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc534020020 con 0x7fc544103fa0 2024-05-03T17:48:55.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:55 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi192 2024-05-03T17:48:55.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:55 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi192 to 8544M 2024-05-03T17:48:55.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:55 smithi122 ceph-mon[29744]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T17:48:55.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:48:55.350+0000 7fc5489e3700 1 -- 172.21.15.122:0/1116450524 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc5440638e0 con 0x7fc5300b2880 2024-05-03T17:48:55.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:55 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi192 2024-05-03T17:48:55.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:55 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi192 to 8544M 2024-05-03T17:48:55.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:55 smithi192 ceph-mon[33753]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T17:48:55.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:55 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi192 2024-05-03T17:48:55.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:55 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi192 to 8544M 2024-05-03T17:48:55.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:55 smithi149 ceph-mon[33409]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T17:48:56.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:56 smithi122 ceph-mon[29744]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:48:56.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:56 smithi122 ceph-mon[29744]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:48:56.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:48:56.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:48:56.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:56.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:56 smithi192 ceph-mon[33753]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:48:56.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:56 smithi192 ceph-mon[33753]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:48:56.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:48:56.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:48:56.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:56.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:56 smithi149 ceph-mon[33409]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:48:56.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:56 smithi149 ceph-mon[33409]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:48:56.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:48:56.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:48:56.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:48:58.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:58 smithi192 ceph-mon[33753]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-03T17:48:58.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:58 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/302141311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4150bfe5-a0e4-4bae-914a-75598f017519"}]: dispatch 2024-05-03T17:48:58.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:58 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/302141311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4150bfe5-a0e4-4bae-914a-75598f017519"}]': finished 2024-05-03T17:48:58.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:58 smithi192 ceph-mon[33753]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T17:48:58.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:58 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:48:58.438 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:58 smithi122 ceph-mon[29744]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-03T17:48:58.438 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:58 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/302141311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4150bfe5-a0e4-4bae-914a-75598f017519"}]: dispatch 2024-05-03T17:48:58.438 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:58 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/302141311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4150bfe5-a0e4-4bae-914a-75598f017519"}]': finished 2024-05-03T17:48:58.438 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:58 smithi122 ceph-mon[29744]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T17:48:58.438 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:58 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:48:58.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:58 smithi149 ceph-mon[33409]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-03T17:48:58.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:58 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/302141311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4150bfe5-a0e4-4bae-914a-75598f017519"}]: dispatch 2024-05-03T17:48:58.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:58 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/302141311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4150bfe5-a0e4-4bae-914a-75598f017519"}]': finished 2024-05-03T17:48:58.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:58 smithi149 ceph-mon[33409]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T17:48:58.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:58 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:48:59.355 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:48:59 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2847567446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:48:59.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:48:59 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2847567446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:48:59.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:48:59 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2847567446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:49:00.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:00 smithi192 ceph-mon[33753]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-03T17:49:00.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:00 smithi122 ceph-mon[29744]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-03T17:49:00.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:00 smithi149 ceph-mon[33409]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-03T17:49:02.366 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:02 smithi122 ceph-mon[29744]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:49:02.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:02 smithi192 ceph-mon[33753]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:49:02.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:02 smithi149 ceph-mon[33409]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:49:04.274 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:04 smithi122 ceph-mon[29744]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:49:04.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:04 smithi149 ceph-mon[33409]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:49:04.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:04 smithi192 ceph-mon[33753]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:49:06.427 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:06 smithi122 ceph-mon[29744]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:49:06.427 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T17:49:06.427 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:06.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:06 smithi149 ceph-mon[33409]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:49:06.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T17:49:06.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:06.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:06 smithi192 ceph-mon[33753]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:49:06.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T17:49:06.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:07.438 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:07 smithi122 ceph-mon[29744]: Deploying daemon osd.6 on smithi122 2024-05-03T17:49:07.438 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:07 smithi149 ceph-mon[33409]: Deploying daemon osd.6 on smithi122 2024-05-03T17:49:07.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:07 smithi192 ceph-mon[33753]: Deploying daemon osd.6 on smithi122 2024-05-03T17:49:08.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:08 smithi122 ceph-mon[29744]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:08.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:08 smithi149 ceph-mon[33409]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:08.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:08 smithi192 ceph-mon[33753]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:10.415 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:10 smithi122 ceph-mon[29744]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:10.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:10 smithi149 ceph-mon[33409]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:10.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:10 smithi192 ceph-mon[33753]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:12.348 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:12 smithi122 ceph-mon[29744]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:12.348 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:12.348 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:12.349 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:12.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:12 smithi149 ceph-mon[33409]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:12.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:12.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:12.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:12.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:12 smithi192 ceph-mon[33753]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:12.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:12.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:12.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:14.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:14 smithi122 ceph-mon[29744]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:14.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:14 smithi149 ceph-mon[33409]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:14.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:14 smithi192 ceph-mon[33753]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:15.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.320+0000 7fc53bfff700 1 -- 172.21.15.122:0/1116450524 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc5440638e0 con 0x7fc5300b2880 2024-05-03T17:49:15.316 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 6 on host 'smithi122' 2024-05-03T17:49:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.322+0000 7fc5489e3700 1 -- 172.21.15.122:0/1116450524 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc5300b2880 msgr2=0x7fc5300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:49:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.322+0000 7fc5489e3700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc5300b2880 0x7fc5300b4d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00b5c0 tx=0x7fc52c005fd0 comp rx=0 tx=0).stop 2024-05-03T17:49:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.322+0000 7fc5489e3700 1 -- 172.21.15.122:0/1116450524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc544103fa0 msgr2=0x7fc544199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:49:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.322+0000 7fc5489e3700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc544103fa0 0x7fc544199700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc53400d9b0 tx=0x7fc53400dcc0 comp rx=0 tx=0).stop 2024-05-03T17:49:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.322+0000 7fc5489e3700 1 -- 172.21.15.122:0/1116450524 shutdown_connections 2024-05-03T17:49:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.322+0000 7fc5489e3700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc5300b2880 0x7fc5300b4d40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.322+0000 7fc5489e3700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc544109200 0x7fc5441a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.322+0000 7fc5489e3700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc544103fa0 0x7fc544199700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.322+0000 7fc5489e3700 1 --2- 172.21.15.122:0/1116450524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5441068d0 0x7fc544199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.322+0000 7fc5489e3700 1 -- 172.21.15.122:0/1116450524 >> 172.21.15.122:0/1116450524 conn(0x7fc5440fd9d0 msgr2=0x7fc5441048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:49:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.323+0000 7fc5489e3700 1 -- 172.21.15.122:0/1116450524 shutdown_connections 2024-05-03T17:49:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:49:15.323+0000 7fc5489e3700 1 -- 172.21.15.122:0/1116450524 wait complete. 2024-05-03T17:49:15.637 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.638 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.638 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:15.638 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:15.638 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.638 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:15 smithi122 ceph-mon[29744]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T17:49:15.638 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:15.638 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.638 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:15.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:15.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:15 smithi149 ceph-mon[33409]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T17:49:15.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:15.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:15.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:15.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:15 smithi192 ceph-mon[33753]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T17:49:15.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:15.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:15.939 DEBUG:teuthology.orchestra.run.smithi122:osd.6> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.6.service 2024-05-03T17:49:15.941 INFO:tasks.cephadm:Deploying osd.7 on smithi149 with /dev/vg_nvme/lv_2... 2024-05-03T17:49:15.941 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T17:49:16.187 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T17:49:16.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:16 smithi149 ceph-mon[33409]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:16.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:16 smithi149 ceph-mon[33409]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T17:49:16.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:16 smithi149 ceph-mon[33409]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T17:49:16.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:16 smithi149 ceph-mon[33409]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:49:16.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:16.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:16 smithi122 ceph-mon[29744]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:16.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:16 smithi122 ceph-mon[29744]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T17:49:16.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:16 smithi122 ceph-mon[29744]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T17:49:16.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:16 smithi122 ceph-mon[29744]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:49:16.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:16.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:16 smithi192 ceph-mon[33753]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:16.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:16 smithi192 ceph-mon[33753]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T17:49:16.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:16 smithi192 ceph-mon[33753]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T17:49:16.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:16 smithi192 ceph-mon[33753]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:49:16.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:17.147 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 17:49:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T17:49:16.892+0000 7fc6bd673700 -1 osd.6 0 waiting for initial osdmap 2024-05-03T17:49:17.147 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 17:49:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T17:49:16.903+0000 7fc6b4c5d700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:49:17.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:17 smithi122 ceph-mon[29744]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:49:17.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:17 smithi122 ceph-mon[29744]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T17:49:17.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:17.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:17 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:17.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:17 smithi122 ceph-mon[29744]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' 2024-05-03T17:49:17.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:17 smithi149 ceph-mon[33409]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:49:17.761 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:17 smithi149 ceph-mon[33409]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T17:49:17.761 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:17.761 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:17 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:17.761 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:17 smithi149 ceph-mon[33409]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' 2024-05-03T17:49:17.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:17 smithi192 ceph-mon[33753]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:49:17.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:17 smithi192 ceph-mon[33753]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T17:49:17.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:17.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:17 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:17.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:17 smithi192 ceph-mon[33753]: from='osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998]' entity='osd.6' 2024-05-03T17:49:18.358 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T17:49:18.358 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T17:49:18.358 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-03T17:49:18.359 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-03T17:49:18.359 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113888 s, 921 MB/s 2024-05-03T17:49:18.359 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-03T17:49:18.667 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:18 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:49:18.667 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:18 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:49:18.667 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:18 smithi149 ceph-mon[33409]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:18.667 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:18.667 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:18 smithi149 ceph-mon[33409]: osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998] boot 2024-05-03T17:49:18.668 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:18 smithi149 ceph-mon[33409]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T17:49:18.668 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:18.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:18 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:49:18.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:18 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:49:18.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:18 smithi122 ceph-mon[29744]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:18.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:18.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:18 smithi122 ceph-mon[29744]: osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998] boot 2024-05-03T17:49:18.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:18 smithi122 ceph-mon[29744]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T17:49:18.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:18.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:18 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:49:18.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:18 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:49:18.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:18 smithi192 ceph-mon[33753]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-03T17:49:18.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:18.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:18 smithi192 ceph-mon[33753]: osd.6 [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998] boot 2024-05-03T17:49:18.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:18 smithi192 ceph-mon[33753]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T17:49:18.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T17:49:19.059 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-05-03T17:49:19.313 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T17:49:20.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:20.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi122 2024-05-03T17:49:20.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T17:49:20.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:20.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:20.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:20.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi122 to 5696M 2024-05-03T17:49:20.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:20.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:20.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:20.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi122 2024-05-03T17:49:20.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T17:49:20.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.214 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:20.214 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:20.214 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:20.214 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi122 to 5696M 2024-05-03T17:49:20.214 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.214 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:20.214 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:20.214 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi122 2024-05-03T17:49:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T17:49:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:20.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi122 to 5696M 2024-05-03T17:49:20.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:20.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:20.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:20.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.970+0000 7efdda254700 1 -- 172.21.15.149:0/82954526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd41050d0 msgr2=0x7efdd410b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:49:20.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.971+0000 7efdda254700 1 --2- 172.21.15.149:0/82954526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd41050d0 0x7efdd410b980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0099c0 tx=0x7efdcc009cd0 comp rx=0 tx=0).stop 2024-05-03T17:49:20.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.971+0000 7efdda254700 1 -- 172.21.15.149:0/82954526 shutdown_connections 2024-05-03T17:49:20.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.971+0000 7efdda254700 1 --2- 172.21.15.149:0/82954526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd41050d0 0x7efdd410b980 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:20.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.971+0000 7efdda254700 1 --2- 172.21.15.149:0/82954526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd4103530 0x7efdd4103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:20.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.971+0000 7efdda254700 1 --2- 172.21.15.149:0/82954526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdd4104730 0x7efdd4104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:20.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.971+0000 7efdda254700 1 -- 172.21.15.149:0/82954526 >> 172.21.15.149:0/82954526 conn(0x7efdd40feaf0 msgr2=0x7efdd4100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:49:20.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.971+0000 7efdda254700 1 -- 172.21.15.149:0/82954526 shutdown_connections 2024-05-03T17:49:20.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.971+0000 7efdda254700 1 -- 172.21.15.149:0/82954526 wait complete. 2024-05-03T17:49:20.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdda254700 1 Processor -- start 2024-05-03T17:49:20.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdda254700 1 -- start start 2024-05-03T17:49:20.970 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdda254700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd4103530 0x7efdd4199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:49:20.970 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdda254700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdd4104730 0x7efdd4199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdda254700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd41050d0 0x7efdd41a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdda254700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efdd410f050 con 0x7efdd4104730 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdda254700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efdd410eed0 con 0x7efdd41050d0 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdda254700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efdd410f1d0 con 0x7efdd4103530 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd41050d0 0x7efdd41a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd41050d0 0x7efdd41a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:55086/0 (socket says 172.21.15.149:55086) 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdd3fff700 1 -- 172.21.15.149:0/1647746493 learned_addr learned my addr 172.21.15.149:0/1647746493 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdd37fe700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd4103530 0x7efdd4199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.972+0000 7efdd2ffd700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdd4104730 0x7efdd4199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.973+0000 7efdd3fff700 1 -- 172.21.15.149:0/1647746493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd4103530 msgr2=0x7efdd4199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.973+0000 7efdd3fff700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd4103530 0x7efdd4199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.973+0000 7efdd3fff700 1 -- 172.21.15.149:0/1647746493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdd4104730 msgr2=0x7efdd4199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.973+0000 7efdd3fff700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdd4104730 0x7efdd4199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:20.971 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.973+0000 7efdd3fff700 1 -- 172.21.15.149:0/1647746493 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd41a1290 con 0x7efdd41050d0 2024-05-03T17:49:20.972 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.973+0000 7efdd3fff700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd41050d0 0x7efdd41a0bf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efdcc003790 tx=0x7efdcc0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:49:20.972 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.973+0000 7efdd0ff9700 1 -- 172.21.15.149:0/1647746493 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc027070 con 0x7efdd41050d0 2024-05-03T17:49:20.972 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.973+0000 7efdda254700 1 -- 172.21.15.149:0/1647746493 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd41a1520 con 0x7efdd41050d0 2024-05-03T17:49:20.972 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.973+0000 7efdd0ff9700 1 -- 172.21.15.149:0/1647746493 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdcc004160 con 0x7efdd41050d0 2024-05-03T17:49:20.972 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.973+0000 7efdd0ff9700 1 -- 172.21.15.149:0/1647746493 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc00f460 con 0x7efdd41050d0 2024-05-03T17:49:20.972 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.973+0000 7efdda254700 1 -- 172.21.15.149:0/1647746493 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd41a1920 con 0x7efdd41050d0 2024-05-03T17:49:20.972 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.974+0000 7efdda254700 1 -- 172.21.15.149:0/1647746493 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd406a190 con 0x7efdd41050d0 2024-05-03T17:49:20.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.975+0000 7efdd0ff9700 1 -- 172.21.15.149:0/1647746493 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7efdcc02a070 con 0x7efdd41050d0 2024-05-03T17:49:20.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.976+0000 7efdd0ff9700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efdbc0b2880 0x7efdbc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:49:20.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.976+0000 7efdd0ff9700 1 -- 172.21.15.149:0/1647746493 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7efdcc0e2d50 con 0x7efdd41050d0 2024-05-03T17:49:20.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.976+0000 7efdd37fe700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efdbc0b2880 0x7efdbc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:49:20.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.977+0000 7efdd37fe700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efdbc0b2880 0x7efdbc0b4d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efdd4104590 tx=0x7efdc400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:49:20.976 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:20.980+0000 7efdd0ff9700 1 -- 172.21.15.149:0/1647746493 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efdcc031020 con 0x7efdd41050d0 2024-05-03T17:49:21.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:21.191+0000 7efdda254700 1 -- 172.21.15.149:0/1647746493 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7efdd40639a0 con 0x7efdbc0b2880 2024-05-03T17:49:22.061 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:21 smithi149 ceph-mon[33409]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:22.062 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:21 smithi149 ceph-mon[33409]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:49:22.062 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:49:22.062 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:49:22.062 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:22.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:21 smithi122 ceph-mon[29744]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:22.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:21 smithi122 ceph-mon[29744]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:49:22.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:49:22.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:49:22.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:22.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:21 smithi192 ceph-mon[33753]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:22.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:21 smithi192 ceph-mon[33753]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:49:22.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:49:22.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:49:22.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:24.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:23 smithi122 ceph-mon[29744]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:24.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:23 smithi149 ceph-mon[33409]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:23 smithi192 ceph-mon[33753]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:25.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:24 smithi122 ceph-mon[29744]: from='client.? 172.21.15.149:0/3661913452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e01812-e511-44a0-a143-2afca510ae8b"}]: dispatch 2024-05-03T17:49:25.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:24 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e01812-e511-44a0-a143-2afca510ae8b"}]: dispatch 2024-05-03T17:49:25.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:24 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71e01812-e511-44a0-a143-2afca510ae8b"}]': finished 2024-05-03T17:49:25.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:24 smithi122 ceph-mon[29744]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T17:49:25.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:24 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:25.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:24 smithi122 ceph-mon[29744]: from='client.? 172.21.15.149:0/1238577038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:49:25.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:24 smithi149 ceph-mon[33409]: from='client.? 172.21.15.149:0/3661913452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e01812-e511-44a0-a143-2afca510ae8b"}]: dispatch 2024-05-03T17:49:25.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:24 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e01812-e511-44a0-a143-2afca510ae8b"}]: dispatch 2024-05-03T17:49:25.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:24 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71e01812-e511-44a0-a143-2afca510ae8b"}]': finished 2024-05-03T17:49:25.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:24 smithi149 ceph-mon[33409]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T17:49:25.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:24 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:25.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:24 smithi149 ceph-mon[33409]: from='client.? 172.21.15.149:0/1238577038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:49:25.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:24 smithi192 ceph-mon[33753]: from='client.? 172.21.15.149:0/3661913452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e01812-e511-44a0-a143-2afca510ae8b"}]: dispatch 2024-05-03T17:49:25.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:24 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e01812-e511-44a0-a143-2afca510ae8b"}]: dispatch 2024-05-03T17:49:25.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:24 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71e01812-e511-44a0-a143-2afca510ae8b"}]': finished 2024-05-03T17:49:25.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:24 smithi192 ceph-mon[33753]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T17:49:25.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:24 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:25.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:24 smithi192 ceph-mon[33753]: from='client.? 172.21.15.149:0/1238577038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:49:26.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:26 smithi149 ceph-mon[33409]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:26.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:26 smithi192 ceph-mon[33753]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:26.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:26 smithi122 ceph-mon[29744]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:28.283 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:28 smithi149 ceph-mon[33409]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:28.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:28 smithi192 ceph-mon[33753]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:28.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:28 smithi122 ceph-mon[29744]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:30.244 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:30 smithi149 ceph-mon[33409]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:30.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:30 smithi192 ceph-mon[33753]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:30.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:30 smithi122 ceph-mon[29744]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:32.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:32 smithi149 ceph-mon[33409]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:32.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T17:49:32.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:32 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:32.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:32 smithi149 ceph-mon[33409]: Deploying daemon osd.7 on smithi149 2024-05-03T17:49:32.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:32 smithi192 ceph-mon[33753]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:32.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T17:49:32.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:32 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:32.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:32 smithi192 ceph-mon[33753]: Deploying daemon osd.7 on smithi149 2024-05-03T17:49:32.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:32 smithi122 ceph-mon[29744]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:32.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T17:49:32.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:32 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:32.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:32 smithi122 ceph-mon[29744]: Deploying daemon osd.7 on smithi149 2024-05-03T17:49:34.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:34 smithi149 ceph-mon[33409]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:34.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:34 smithi192 ceph-mon[33753]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:34.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:34 smithi122 ceph-mon[29744]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:36.210 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:36 smithi149 ceph-mon[33409]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:36.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:36 smithi192 ceph-mon[33753]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:36.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:36 smithi122 ceph-mon[29744]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:38.402 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:38 smithi149 ceph-mon[33409]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:38.402 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:38 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:38.402 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:38 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:38.402 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:38 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:38.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:38 smithi122 ceph-mon[29744]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:38.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:38 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:38.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:38 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:38.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:38 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:38.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:38 smithi192 ceph-mon[33753]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:38.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:38 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:38.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:38 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:38.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:38 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:40.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:40 smithi149 ceph-mon[33409]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:40.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:40 smithi122 ceph-mon[29744]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:40.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:40 smithi192 ceph-mon[33753]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:41.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.165+0000 7efdd0ff9700 1 -- 172.21.15.149:0/1647746493 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efdd40639a0 con 0x7efdbc0b2880 2024-05-03T17:49:41.160 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 7 on host 'smithi149' 2024-05-03T17:49:41.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.167+0000 7efdda254700 1 -- 172.21.15.149:0/1647746493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efdbc0b2880 msgr2=0x7efdbc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:49:41.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.167+0000 7efdda254700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efdbc0b2880 0x7efdbc0b4d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efdd4104590 tx=0x7efdc400d040 comp rx=0 tx=0).stop 2024-05-03T17:49:41.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.167+0000 7efdda254700 1 -- 172.21.15.149:0/1647746493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd41050d0 msgr2=0x7efdd41a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:49:41.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.167+0000 7efdda254700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd41050d0 0x7efdd41a0bf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efdcc003790 tx=0x7efdcc0057c0 comp rx=0 tx=0).stop 2024-05-03T17:49:41.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.167+0000 7efdda254700 1 -- 172.21.15.149:0/1647746493 shutdown_connections 2024-05-03T17:49:41.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.167+0000 7efdda254700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7efdbc0b2880 0x7efdbc0b4d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:41.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.167+0000 7efdda254700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd41050d0 0x7efdd41a0bf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:41.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.167+0000 7efdda254700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd4103530 0x7efdd4199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:41.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.167+0000 7efdda254700 1 --2- 172.21.15.149:0/1647746493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdd4104730 0x7efdd4199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:41.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.167+0000 7efdda254700 1 -- 172.21.15.149:0/1647746493 >> 172.21.15.149:0/1647746493 conn(0x7efdd40feaf0 msgr2=0x7efdd4107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:49:41.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.168+0000 7efdda254700 1 -- 172.21.15.149:0/1647746493 shutdown_connections 2024-05-03T17:49:41.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:49:41.168+0000 7efdda254700 1 -- 172.21.15.149:0/1647746493 wait complete. 2024-05-03T17:49:41.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:41.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:41.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:41 smithi122 ceph-mon[29744]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T17:49:41.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:41 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:41.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:41 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:41 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:41 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:41.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:41 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:41.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:41 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:41 smithi149 ceph-mon[33409]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T17:49:41.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:41 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:41 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:41 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:41 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:41 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:41 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:41 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:41 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:41 smithi192 ceph-mon[33753]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T17:49:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:41 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:41 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:41.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:41 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:49:41.857 DEBUG:teuthology.orchestra.run.smithi149:osd.7> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.7.service 2024-05-03T17:49:41.860 INFO:tasks.cephadm:Deploying osd.8 on smithi192 with /dev/vg_nvme/lv_2... 2024-05-03T17:49:41.860 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T17:49:42.107 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:49:42.351 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:42 smithi149 ceph-mon[33409]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:42.351 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:42 smithi149 ceph-mon[33409]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T17:49:42.351 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:42 smithi149 ceph-mon[33409]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T17:49:42.351 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:42 smithi149 ceph-mon[33409]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:49:42.351 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:42.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:42 smithi192 ceph-mon[33753]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:42.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:42 smithi192 ceph-mon[33753]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T17:49:42.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:42 smithi192 ceph-mon[33753]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T17:49:42.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:42 smithi192 ceph-mon[33753]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:49:42.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:42.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:42 smithi122 ceph-mon[29744]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:42.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:42 smithi122 ceph-mon[29744]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T17:49:42.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:42 smithi122 ceph-mon[29744]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T17:49:42.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:42 smithi122 ceph-mon[29744]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:49:42.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:43.037 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 17:49:42 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T17:49:42.698+0000 7f3e444e4700 -1 osd.7 0 waiting for initial osdmap 2024-05-03T17:49:43.037 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 17:49:42 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T17:49:42.707+0000 7f3e3c2cf700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:49:43.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:43 smithi149 ceph-mon[33409]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:49:43.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:43 smithi149 ceph-mon[33409]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T17:49:43.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:43.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:43.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:43 smithi149 ceph-mon[33409]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' 2024-05-03T17:49:43.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:43 smithi149 ceph-mon[33409]: osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549] boot 2024-05-03T17:49:43.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:43 smithi149 ceph-mon[33409]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T17:49:43.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:43 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:43.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:43 smithi192 ceph-mon[33753]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:49:43.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:43 smithi192 ceph-mon[33753]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T17:49:43.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:43.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:43.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:43 smithi192 ceph-mon[33753]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' 2024-05-03T17:49:43.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:43 smithi192 ceph-mon[33753]: osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549] boot 2024-05-03T17:49:43.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:43 smithi192 ceph-mon[33753]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T17:49:43.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:43 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:43.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:43 smithi122 ceph-mon[29744]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:49:43.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:43 smithi122 ceph-mon[29744]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T17:49:43.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:43.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:43.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:43 smithi122 ceph-mon[29744]: from='osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549]' entity='osd.7' 2024-05-03T17:49:43.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:43 smithi122 ceph-mon[29744]: osd.7 [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549] boot 2024-05-03T17:49:43.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:43 smithi122 ceph-mon[29744]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T17:49:43.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:43 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T17:49:44.271 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T17:49:44.271 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T17:49:44.271 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-05-03T17:49:44.271 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-05-03T17:49:44.271 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113566 s, 923 MB/s 2024-05-03T17:49:44.272 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-05-03T17:49:44.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:44 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:49:44.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:44 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:49:44.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:44 smithi149 ceph-mon[33409]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:44.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:44 smithi149 ceph-mon[33409]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T17:49:44.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:44 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:49:44.573 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:44 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:49:44.573 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:44 smithi192 ceph-mon[33753]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:44.573 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:44 smithi192 ceph-mon[33753]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T17:49:44.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:44 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:49:44.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:44 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:49:44.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:44 smithi122 ceph-mon[29744]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-03T17:49:44.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:44 smithi122 ceph-mon[29744]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T17:49:44.954 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2024-05-03T17:49:45.205 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:49:45.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:49:45.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T17:49:45.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi149 2024-05-03T17:49:45.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:45.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:45.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:45.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:45.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:45.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi149 to 1938M 2024-05-03T17:49:45.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:45.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:45.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:45.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:46.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:49:46.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T17:49:46.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi149 2024-05-03T17:49:46.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:46.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:46.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:46.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:46.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:46.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi149 to 1938M 2024-05-03T17:49:46.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:46.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:46.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:46.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:46.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:49:46.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T17:49:46.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi149 2024-05-03T17:49:46.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:46.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:46.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:46.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:46.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:49:46.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi149 to 1938M 2024-05-03T17:49:46.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:46.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:46.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:49:46.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:49:46.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.828+0000 7fbd8dafc700 1 -- 172.21.15.192:0/1813937978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd88103500 msgr2=0x7fbd88103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:49:46.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.828+0000 7fbd8dafc700 1 --2- 172.21.15.192:0/1813937978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd88103500 0x7fbd88103920 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009990 tx=0x7fbd78009ca0 comp rx=0 tx=0).stop 2024-05-03T17:49:46.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.828+0000 7fbd8dafc700 1 -- 172.21.15.192:0/1813937978 shutdown_connections 2024-05-03T17:49:46.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.828+0000 7fbd8dafc700 1 --2- 172.21.15.192:0/1813937978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd881050a0 0x7fbd8810b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:46.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.828+0000 7fbd8dafc700 1 --2- 172.21.15.192:0/1813937978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd88103500 0x7fbd88103920 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:46.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.828+0000 7fbd8dafc700 1 --2- 172.21.15.192:0/1813937978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd88104700 0x7fbd88104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:46.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.828+0000 7fbd8dafc700 1 -- 172.21.15.192:0/1813937978 >> 172.21.15.192:0/1813937978 conn(0x7fbd880feaa0 msgr2=0x7fbd88100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:49:46.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.828+0000 7fbd8dafc700 1 -- 172.21.15.192:0/1813937978 shutdown_connections 2024-05-03T17:49:46.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.828+0000 7fbd8dafc700 1 -- 172.21.15.192:0/1813937978 wait complete. 2024-05-03T17:49:46.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.828+0000 7fbd8dafc700 1 Processor -- start 2024-05-03T17:49:46.826 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd8dafc700 1 -- start start 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd8dafc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd88103500 0x7fbd88199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd8dafc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd88104700 0x7fbd88199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd8dafc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd881050a0 0x7fbd881a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd8dafc700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8810f000 con 0x7fbd881050a0 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd8dafc700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8810ee80 con 0x7fbd88104700 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd8dafc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8810f180 con 0x7fbd88103500 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd877fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd88103500 0x7fbd88199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd877fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd88103500 0x7fbd88199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:41942/0 (socket says 172.21.15.192:41942) 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd877fe700 1 -- 172.21.15.192:0/2188843845 learned_addr learned my addr 172.21.15.192:0/2188843845 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd87fff700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd881050a0 0x7fbd881a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd86ffd700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd88104700 0x7fbd88199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:49:46.827 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd877fe700 1 -- 172.21.15.192:0/2188843845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd88104700 msgr2=0x7fbd88199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd877fe700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd88104700 0x7fbd88199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd877fe700 1 -- 172.21.15.192:0/2188843845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd881050a0 msgr2=0x7fbd881a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd877fe700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd881050a0 0x7fbd881a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd877fe700 1 -- 172.21.15.192:0/2188843845 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd881a1350 con 0x7fbd88103500 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd86ffd700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd88104700 0x7fbd88199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd87fff700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd881050a0 0x7fbd881a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.829+0000 7fbd877fe700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd88103500 0x7fbd88199740 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbd780047b0 tx=0x7fbd78009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.830+0000 7fbd84ff9700 1 -- 172.21.15.192:0/2188843845 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78026070 con 0x7fbd88103500 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.830+0000 7fbd8dafc700 1 -- 172.21.15.192:0/2188843845 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd881a15e0 con 0x7fbd88103500 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.830+0000 7fbd84ff9700 1 -- 172.21.15.192:0/2188843845 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd780047e0 con 0x7fbd88103500 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.830+0000 7fbd84ff9700 1 -- 172.21.15.192:0/2188843845 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78005320 con 0x7fbd88103500 2024-05-03T17:49:46.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.830+0000 7fbd8dafc700 1 -- 172.21.15.192:0/2188843845 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd881a1a90 con 0x7fbd88103500 2024-05-03T17:49:46.829 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.831+0000 7fbd8dafc700 1 -- 172.21.15.192:0/2188843845 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8806a190 con 0x7fbd88103500 2024-05-03T17:49:46.829 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.832+0000 7fbd84ff9700 1 -- 172.21.15.192:0/2188843845 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fbd78029070 con 0x7fbd88103500 2024-05-03T17:49:46.829 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.833+0000 7fbd84ff9700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbd700b2880 0x7fbd700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:49:46.830 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.833+0000 7fbd84ff9700 1 -- 172.21.15.192:0/2188843845 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7fbd780e26e0 con 0x7fbd88103500 2024-05-03T17:49:46.830 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.833+0000 7fbd86ffd700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbd700b2880 0x7fbd700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:49:46.830 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.834+0000 7fbd86ffd700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbd700b2880 0x7fbd700b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c007900 tx=0x7fbd7c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:49:46.832 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:46.837+0000 7fbd84ff9700 1 -- 172.21.15.192:0/2188843845 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbd78030020 con 0x7fbd88103500 2024-05-03T17:49:47.043 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:49:47.048+0000 7fbd8dafc700 1 -- 172.21.15.192:0/2188843845 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fbd880638e0 con 0x7fbd700b2880 2024-05-03T17:49:47.917 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:47 smithi192 ceph-mon[33753]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:49:47.917 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:47 smithi192 ceph-mon[33753]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:49:47.917 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:47 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:49:47.917 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:47 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:49:47.917 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:47 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:48.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:47 smithi149 ceph-mon[33409]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:49:48.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:47 smithi149 ceph-mon[33409]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:49:48.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:47 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:49:48.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:47 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:49:48.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:47 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:48.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:47 smithi122 ceph-mon[29744]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:49:48.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:47 smithi122 ceph-mon[29744]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:49:48.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:47 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:49:48.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:47 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:49:48.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:47 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:50.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:49 smithi149 ceph-mon[33409]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T17:49:50.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:49 smithi192 ceph-mon[33753]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T17:49:50.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:49 smithi122 ceph-mon[29744]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T17:49:50.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:50 smithi122 ceph-mon[29744]: from='client.? 172.21.15.192:0/2056600684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5294516d-fac6-41d0-a635-6bc1d7eca585"}]: dispatch 2024-05-03T17:49:50.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:50 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5294516d-fac6-41d0-a635-6bc1d7eca585"}]: dispatch 2024-05-03T17:49:50.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:50 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5294516d-fac6-41d0-a635-6bc1d7eca585"}]': finished 2024-05-03T17:49:50.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:50 smithi122 ceph-mon[29744]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T17:49:50.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:49:50.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:50 smithi122 ceph-mon[29744]: from='client.? 172.21.15.192:0/1083855390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:49:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:50 smithi149 ceph-mon[33409]: from='client.? 172.21.15.192:0/2056600684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5294516d-fac6-41d0-a635-6bc1d7eca585"}]: dispatch 2024-05-03T17:49:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:50 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5294516d-fac6-41d0-a635-6bc1d7eca585"}]: dispatch 2024-05-03T17:49:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:50 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5294516d-fac6-41d0-a635-6bc1d7eca585"}]': finished 2024-05-03T17:49:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:50 smithi149 ceph-mon[33409]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T17:49:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:50 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:49:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:50 smithi149 ceph-mon[33409]: from='client.? 172.21.15.192:0/1083855390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:49:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:50 smithi192 ceph-mon[33753]: from='client.? 172.21.15.192:0/2056600684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5294516d-fac6-41d0-a635-6bc1d7eca585"}]: dispatch 2024-05-03T17:49:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:50 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5294516d-fac6-41d0-a635-6bc1d7eca585"}]: dispatch 2024-05-03T17:49:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:50 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5294516d-fac6-41d0-a635-6bc1d7eca585"}]': finished 2024-05-03T17:49:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:50 smithi192 ceph-mon[33753]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T17:49:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:50 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:49:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:50 smithi192 ceph-mon[33753]: from='client.? 172.21.15.192:0/1083855390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:49:52.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:51 smithi192 ceph-mon[33753]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:49:52.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:51 smithi122 ceph-mon[29744]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:49:52.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:51 smithi149 ceph-mon[33409]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:49:54.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:53 smithi192 ceph-mon[33753]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:49:54.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:53 smithi122 ceph-mon[29744]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:49:54.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:53 smithi149 ceph-mon[33409]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:49:55.932 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:55 smithi192 ceph-mon[33753]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:49:56.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:55 smithi122 ceph-mon[29744]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:49:56.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:55 smithi149 ceph-mon[33409]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:49:57.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:57 smithi192 ceph-mon[33753]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T17:49:57.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T17:49:57.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:57.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:57 smithi192 ceph-mon[33753]: Deploying daemon osd.8 on smithi192 2024-05-03T17:49:58.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:57 smithi122 ceph-mon[29744]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T17:49:58.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T17:49:58.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:58.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:57 smithi122 ceph-mon[29744]: Deploying daemon osd.8 on smithi192 2024-05-03T17:49:58.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:57 smithi149 ceph-mon[33409]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T17:49:58.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T17:49:58.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:49:58.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:57 smithi149 ceph-mon[33409]: Deploying daemon osd.8 on smithi192 2024-05-03T17:50:00.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:49:59 smithi192 ceph-mon[33753]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:00.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:49:59 smithi122 ceph-mon[29744]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:00.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:49:59 smithi149 ceph-mon[33409]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:00.957 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:00 smithi192 ceph-mon[33753]: overall HEALTH_OK 2024-05-03T17:50:01.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:00 smithi122 ceph-mon[29744]: overall HEALTH_OK 2024-05-03T17:50:01.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:00 smithi149 ceph-mon[33409]: overall HEALTH_OK 2024-05-03T17:50:02.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:01 smithi192 ceph-mon[33753]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:02.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:01 smithi122 ceph-mon[29744]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:02.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:01 smithi149 ceph-mon[33409]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:03.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:03 smithi122 ceph-mon[29744]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:03.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:03.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:03.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:04.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:03 smithi149 ceph-mon[33409]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:04.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:04.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:04.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:04.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:03 smithi192 ceph-mon[33753]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:04.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:04.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:04.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:05.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:05 smithi192 ceph-mon[33753]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:05.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:05 smithi122 ceph-mon[29744]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:06.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:05 smithi149 ceph-mon[33409]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:06.759 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.763+0000 7fbd84ff9700 1 -- 172.21.15.192:0/2188843845 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbd880638e0 con 0x7fbd700b2880 2024-05-03T17:50:06.759 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 8 on host 'smithi192' 2024-05-03T17:50:06.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.765+0000 7fbd8dafc700 1 -- 172.21.15.192:0/2188843845 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbd700b2880 msgr2=0x7fbd700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:06.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.765+0000 7fbd8dafc700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbd700b2880 0x7fbd700b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c007900 tx=0x7fbd7c008040 comp rx=0 tx=0).stop 2024-05-03T17:50:06.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.765+0000 7fbd8dafc700 1 -- 172.21.15.192:0/2188843845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd88103500 msgr2=0x7fbd88199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:06.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.765+0000 7fbd8dafc700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd88103500 0x7fbd88199740 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbd780047b0 tx=0x7fbd78009580 comp rx=0 tx=0).stop 2024-05-03T17:50:06.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.765+0000 7fbd8dafc700 1 -- 172.21.15.192:0/2188843845 shutdown_connections 2024-05-03T17:50:06.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.765+0000 7fbd8dafc700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbd700b2880 0x7fbd700b4d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:06.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.765+0000 7fbd8dafc700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd881050a0 0x7fbd881a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:06.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.765+0000 7fbd8dafc700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd88103500 0x7fbd88199740 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:06.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.765+0000 7fbd8dafc700 1 --2- 172.21.15.192:0/2188843845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd88104700 0x7fbd88199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:06.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.765+0000 7fbd8dafc700 1 -- 172.21.15.192:0/2188843845 >> 172.21.15.192:0/2188843845 conn(0x7fbd880feaa0 msgr2=0x7fbd88107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:50:06.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.766+0000 7fbd8dafc700 1 -- 172.21.15.192:0/2188843845 shutdown_connections 2024-05-03T17:50:06.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:50:06.766+0000 7fbd8dafc700 1 -- 172.21.15.192:0/2188843845 wait complete. 2024-05-03T17:50:07.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:07.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:07.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:06 smithi192 ceph-mon[33753]: from='osd.8 [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T17:50:07.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:06 smithi192 ceph-mon[33753]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T17:50:07.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:07.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:07.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:07.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:06 smithi122 ceph-mon[29744]: from='osd.8 [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T17:50:07.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:06 smithi122 ceph-mon[29744]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T17:50:07.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:07.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:07.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:07.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:06 smithi149 ceph-mon[33409]: from='osd.8 [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T17:50:07.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:06 smithi149 ceph-mon[33409]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T17:50:07.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:07.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:07.419 DEBUG:teuthology.orchestra.run.smithi192:osd.8> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.8.service 2024-05-03T17:50:07.422 INFO:tasks.cephadm:Deploying osd.9 on smithi122 with /dev/vg_nvme/lv_1... 2024-05-03T17:50:07.422 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T17:50:07.676 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:50:07.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:07 smithi192 ceph-mon[33753]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:07.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:07 smithi192 ceph-mon[33753]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T17:50:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:07 smithi192 ceph-mon[33753]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T17:50:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:07 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:07 smithi192 ceph-mon[33753]: from='osd.8 [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:50:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:07 smithi192 ceph-mon[33753]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:50:08.118 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:07 smithi122 ceph-mon[29744]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:08.118 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:07 smithi122 ceph-mon[29744]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T17:50:08.118 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:07 smithi122 ceph-mon[29744]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T17:50:08.118 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:07 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:08.119 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:07 smithi122 ceph-mon[29744]: from='osd.8 [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:50:08.119 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:07 smithi122 ceph-mon[29744]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:50:08.188 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:07 smithi149 ceph-mon[33409]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:08.189 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:07 smithi149 ceph-mon[33409]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T17:50:08.189 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:07 smithi149 ceph-mon[33409]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T17:50:08.189 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:07 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:08.189 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:07 smithi149 ceph-mon[33409]: from='osd.8 [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:50:08.189 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:07 smithi149 ceph-mon[33409]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:50:08.616 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 17:50:08 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T17:50:08.285+0000 7f2848826700 -1 osd.8 0 waiting for initial osdmap 2024-05-03T17:50:08.616 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 17:50:08 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T17:50:08.294+0000 7f283fe10700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:50:08.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:08 smithi192 ceph-mon[33753]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:50:08.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:08 smithi192 ceph-mon[33753]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T17:50:08.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:08 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:08.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:08 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:08.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:08 smithi192 ceph-mon[33753]: from='osd.8 ' entity='osd.8' 2024-05-03T17:50:08.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:08 smithi192 ceph-mon[33753]: osd.8 [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744] boot 2024-05-03T17:50:08.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:08 smithi192 ceph-mon[33753]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T17:50:08.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:08 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:09.143 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:08 smithi122 ceph-mon[29744]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:50:09.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:08 smithi122 ceph-mon[29744]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T17:50:09.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:08 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:09.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:08 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:09.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:08 smithi122 ceph-mon[29744]: from='osd.8 ' entity='osd.8' 2024-05-03T17:50:09.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:08 smithi122 ceph-mon[29744]: osd.8 [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744] boot 2024-05-03T17:50:09.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:08 smithi122 ceph-mon[29744]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T17:50:09.144 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:08 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:09.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:08 smithi149 ceph-mon[33409]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:50:09.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:08 smithi149 ceph-mon[33409]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T17:50:09.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:08 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:09.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:08 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:09.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:08 smithi149 ceph-mon[33409]: from='osd.8 ' entity='osd.8' 2024-05-03T17:50:09.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:08 smithi149 ceph-mon[33409]: osd.8 [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744] boot 2024-05-03T17:50:09.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:08 smithi149 ceph-mon[33409]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T17:50:09.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:08 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T17:50:09.761 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T17:50:09.761 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T17:50:09.761 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-05-03T17:50:09.762 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-05-03T17:50:09.762 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114285 s, 918 MB/s 2024-05-03T17:50:09.762 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-05-03T17:50:10.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:10 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:50:10.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:10 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:50:10.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:10 smithi192 ceph-mon[33753]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:10.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:10 smithi192 ceph-mon[33753]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T17:50:10.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:10 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:50:10.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:10 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:50:10.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:10 smithi122 ceph-mon[29744]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:10.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:10 smithi122 ceph-mon[29744]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T17:50:10.510 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_1 2024-05-03T17:50:10.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:10 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:50:10.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:10 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:50:10.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:10 smithi149 ceph-mon[33409]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-03T17:50:10.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:10 smithi149 ceph-mon[33409]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T17:50:10.753 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:50:11.951 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:11.951 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T17:50:11.951 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi192 2024-05-03T17:50:11.951 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:11.951 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:11.951 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:11.951 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:11.952 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:11.952 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi192 to 5696M 2024-05-03T17:50:11.952 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:11.952 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:11.952 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:11.952 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:12.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi192 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi192 to 5696M 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:12.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:12.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:12.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T17:50:12.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi192 2024-05-03T17:50:12.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:12.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:12.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:12.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:12.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:12.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi192 to 5696M 2024-05-03T17:50:12.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:12.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:12.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:12.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:12.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.479+0000 7f2fa6a47700 1 -- 172.21.15.122:0/943621489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa01050a0 msgr2=0x7f2fa010b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:12.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.479+0000 7f2fa6a47700 1 --2- 172.21.15.122:0/943621489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa01050a0 0x7f2fa010b950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800b0d0 tx=0x7f2f9800b3e0 comp rx=0 tx=0).stop 2024-05-03T17:50:12.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.479+0000 7f2fa6a47700 1 -- 172.21.15.122:0/943621489 shutdown_connections 2024-05-03T17:50:12.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.479+0000 7f2fa6a47700 1 --2- 172.21.15.122:0/943621489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa01050a0 0x7f2fa010b950 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:12.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.479+0000 7f2fa6a47700 1 --2- 172.21.15.122:0/943621489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0103500 0x7f2fa0103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:12.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.479+0000 7f2fa6a47700 1 --2- 172.21.15.122:0/943621489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa0104700 0x7f2fa0104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:12.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.479+0000 7f2fa6a47700 1 -- 172.21.15.122:0/943621489 >> 172.21.15.122:0/943621489 conn(0x7f2fa00feaa0 msgr2=0x7f2fa0100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:50:12.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.479+0000 7f2fa6a47700 1 -- 172.21.15.122:0/943621489 shutdown_connections 2024-05-03T17:50:12.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.479+0000 7f2fa6a47700 1 -- 172.21.15.122:0/943621489 wait complete. 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.479+0000 7f2fa6a47700 1 Processor -- start 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.479+0000 7f2fa6a47700 1 -- start start 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2fa6a47700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0103500 0x7f2fa0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2fa6a47700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa0104700 0x7f2fa0199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2fa6a47700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa01050a0 0x7f2fa01a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2fa6a47700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa010ef80 con 0x7f2fa0104700 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2fa6a47700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa010ee00 con 0x7f2fa0103500 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2fa6a47700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa010f100 con 0x7f2fa01050a0 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2f9f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa0104700 0x7f2fa0199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2f9f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa0104700 0x7f2fa0199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48100/0 (socket says 172.21.15.122:48100) 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2f9f7fe700 1 -- 172.21.15.122:0/2011686990 learned_addr learned my addr 172.21.15.122:0/2011686990 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2f9ffff700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0103500 0x7f2fa0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2fa4fe4700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa01050a0 0x7f2fa01a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2f9f7fe700 1 -- 172.21.15.122:0/2011686990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa01050a0 msgr2=0x7f2fa01a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2f9f7fe700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa01050a0 0x7f2fa01a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2f9f7fe700 1 -- 172.21.15.122:0/2011686990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0103500 msgr2=0x7f2fa0199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2f9f7fe700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0103500 0x7f2fa0199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2f9f7fe700 1 -- 172.21.15.122:0/2011686990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa01a11a0 con 0x7f2fa0104700 2024-05-03T17:50:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.480+0000 7f2f9f7fe700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa0104700 0x7f2fa0199a80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c9e0 tx=0x7f2f9400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:50:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.481+0000 7f2f9d7fa700 1 -- 172.21.15.122:0/2011686990 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94007ac0 con 0x7f2fa0104700 2024-05-03T17:50:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.481+0000 7f2f9d7fa700 1 -- 172.21.15.122:0/2011686990 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f94007c60 con 0x7f2fa0104700 2024-05-03T17:50:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.481+0000 7f2fa6a47700 1 -- 172.21.15.122:0/2011686990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa01a1490 con 0x7f2fa0104700 2024-05-03T17:50:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.481+0000 7f2f9d7fa700 1 -- 172.21.15.122:0/2011686990 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94018660 con 0x7f2fa0104700 2024-05-03T17:50:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.481+0000 7f2fa6a47700 1 -- 172.21.15.122:0/2011686990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa01a19d0 con 0x7f2fa0104700 2024-05-03T17:50:12.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.482+0000 7f2fa6a47700 1 -- 172.21.15.122:0/2011686990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa006a190 con 0x7f2fa0104700 2024-05-03T17:50:12.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.483+0000 7f2f9d7fa700 1 -- 172.21.15.122:0/2011686990 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f2f9400f450 con 0x7f2fa0104700 2024-05-03T17:50:12.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.483+0000 7f2f9d7fa700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2f880b2900 0x7f2f880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:50:12.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.484+0000 7f2f9ffff700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2f880b2900 0x7f2f880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:50:12.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.484+0000 7f2f9d7fa700 1 -- 172.21.15.122:0/2011686990 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f2f940e1d40 con 0x7f2fa0104700 2024-05-03T17:50:12.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.484+0000 7f2f9ffff700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2f880b2900 0x7f2f880b4dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f90005fd0 tx=0x7f2f90005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:50:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.487+0000 7f2f9d7fa700 1 -- 172.21.15.122:0/2011686990 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2f94014030 con 0x7f2fa0104700 2024-05-03T17:50:12.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:12.701+0000 7f2fa6a47700 1 -- 172.21.15.122:0/2011686990 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2fa00638e0 con 0x7f2f880b2900 2024-05-03T17:50:12.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:50:12.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:50:12.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:13.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:50:13.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:50:13.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:13.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:50:13.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:50:13.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:13.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:13 smithi122 ceph-mon[29744]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:13.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:13 smithi122 ceph-mon[29744]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:50:14.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:13 smithi149 ceph-mon[33409]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:14.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:13 smithi149 ceph-mon[33409]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:50:14.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:13 smithi192 ceph-mon[33753]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:14.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:13 smithi192 ceph-mon[33753]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:50:16.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:15 smithi149 ceph-mon[33409]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T17:50:16.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:15 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/208956850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126"}]: dispatch 2024-05-03T17:50:16.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:15 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/208956850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126"}]': finished 2024-05-03T17:50:16.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:15 smithi149 ceph-mon[33409]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T17:50:16.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:16.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:15 smithi192 ceph-mon[33753]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T17:50:16.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:15 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/208956850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126"}]: dispatch 2024-05-03T17:50:16.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:15 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/208956850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126"}]': finished 2024-05-03T17:50:16.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:15 smithi192 ceph-mon[33753]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T17:50:16.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:16.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:15 smithi122 ceph-mon[29744]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T17:50:16.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:15 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/208956850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126"}]: dispatch 2024-05-03T17:50:16.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:15 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/208956850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126"}]': finished 2024-05-03T17:50:16.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:15 smithi122 ceph-mon[29744]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T17:50:16.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:17.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:16 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2716878668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:50:17.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:16 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2716878668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:50:17.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:16 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2716878668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:50:18.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:17 smithi149 ceph-mon[33409]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:50:18.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:17 smithi192 ceph-mon[33753]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:50:18.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:17 smithi122 ceph-mon[29744]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-03T17:50:19.882 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:19 smithi122 ceph-mon[29744]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:50:20.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:19 smithi149 ceph-mon[33409]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:50:20.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:19 smithi192 ceph-mon[33753]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T17:50:21.833 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:21 smithi122 ceph-mon[29744]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:50:22.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:21 smithi149 ceph-mon[33409]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:50:22.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:21 smithi192 ceph-mon[33753]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T17:50:23.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:23 smithi122 ceph-mon[29744]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T17:50:23.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T17:50:23.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:23.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:23 smithi122 ceph-mon[29744]: Deploying daemon osd.9 on smithi122 2024-05-03T17:50:24.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:23 smithi149 ceph-mon[33409]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T17:50:24.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T17:50:24.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:24.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:23 smithi149 ceph-mon[33409]: Deploying daemon osd.9 on smithi122 2024-05-03T17:50:24.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:23 smithi192 ceph-mon[33753]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T17:50:24.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T17:50:24.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:24.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:23 smithi192 ceph-mon[33753]: Deploying daemon osd.9 on smithi122 2024-05-03T17:50:26.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:25 smithi149 ceph-mon[33409]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:25 smithi192 ceph-mon[33753]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:26.174 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:25 smithi122 ceph-mon[29744]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:27.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:27 smithi122 ceph-mon[29744]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:28.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:27 smithi149 ceph-mon[33409]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:28.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:27 smithi192 ceph-mon[33753]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:30.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:29 smithi122 ceph-mon[29744]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:30.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:30.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:30.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:30.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:29 smithi149 ceph-mon[33409]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:30.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:30.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:30.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:30.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:29 smithi192 ceph-mon[33753]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:30.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:30.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:30.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:32.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:31 smithi122 ceph-mon[29744]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:32.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:31 smithi149 ceph-mon[33409]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:32.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:31 smithi192 ceph-mon[33753]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:33.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.056+0000 7f2f9d7fa700 1 -- 172.21.15.122:0/2011686990 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2fa00638e0 con 0x7f2f880b2900 2024-05-03T17:50:33.052 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 9 on host 'smithi122' 2024-05-03T17:50:33.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.058+0000 7f2fa6a47700 1 -- 172.21.15.122:0/2011686990 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2f880b2900 msgr2=0x7f2f880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:33.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.058+0000 7f2fa6a47700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2f880b2900 0x7f2f880b4dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f90005fd0 tx=0x7f2f90005dd0 comp rx=0 tx=0).stop 2024-05-03T17:50:33.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.058+0000 7f2fa6a47700 1 -- 172.21.15.122:0/2011686990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa0104700 msgr2=0x7f2fa0199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:33.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.058+0000 7f2fa6a47700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa0104700 0x7f2fa0199a80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c9e0 tx=0x7f2f9400cda0 comp rx=0 tx=0).stop 2024-05-03T17:50:33.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.059+0000 7f2fa6a47700 1 -- 172.21.15.122:0/2011686990 shutdown_connections 2024-05-03T17:50:33.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.059+0000 7f2fa6a47700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f2f880b2900 0x7f2f880b4dc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:33.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.059+0000 7f2fa6a47700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa01050a0 0x7f2fa01a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:33.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.059+0000 7f2fa6a47700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0103500 0x7f2fa0199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:33.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.059+0000 7f2fa6a47700 1 --2- 172.21.15.122:0/2011686990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa0104700 0x7f2fa0199a80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:33.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.059+0000 7f2fa6a47700 1 -- 172.21.15.122:0/2011686990 >> 172.21.15.122:0/2011686990 conn(0x7f2fa00feaa0 msgr2=0x7f2fa0107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:50:33.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.059+0000 7f2fa6a47700 1 -- 172.21.15.122:0/2011686990 shutdown_connections 2024-05-03T17:50:33.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:50:33.059+0000 7f2fa6a47700 1 -- 172.21.15.122:0/2011686990 wait complete. 2024-05-03T17:50:33.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:33.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:33.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:33 smithi122 ceph-mon[29744]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T17:50:33.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.368 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:33.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:33.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:33.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:33 smithi149 ceph-mon[33409]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T17:50:33.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:33.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:33.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:33.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:33 smithi192 ceph-mon[33753]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T17:50:33.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:33.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:33.704 DEBUG:teuthology.orchestra.run.smithi122:osd.9> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.9.service 2024-05-03T17:50:33.707 INFO:tasks.cephadm:Deploying osd.10 on smithi149 with /dev/vg_nvme/lv_1... 2024-05-03T17:50:33.707 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T17:50:33.955 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T17:50:34.264 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:34 smithi122 ceph-mon[29744]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:34.265 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:34 smithi122 ceph-mon[29744]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T17:50:34.265 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:34 smithi122 ceph-mon[29744]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T17:50:34.265 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:34 smithi122 ceph-mon[29744]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:50:34.265 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:34 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:34.328 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:34 smithi149 ceph-mon[33409]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:34.329 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:34 smithi149 ceph-mon[33409]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T17:50:34.329 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:34 smithi149 ceph-mon[33409]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T17:50:34.329 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:34 smithi149 ceph-mon[33409]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:50:34.329 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:34 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:34 smithi192 ceph-mon[33753]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:34 smithi192 ceph-mon[33753]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T17:50:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:34 smithi192 ceph-mon[33753]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T17:50:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:34 smithi192 ceph-mon[33753]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-05-03T17:50:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:34 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:34.953 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 17:50:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T17:50:34.697+0000 7f0cfd4af700 -1 osd.9 0 waiting for initial osdmap 2024-05-03T17:50:34.953 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 17:50:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T17:50:34.708+0000 7f0cf529a700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:50:35.383 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:35 smithi149 ceph-mon[33409]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:50:35.384 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:35 smithi149 ceph-mon[33409]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T17:50:35.384 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:35 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:35.384 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:35 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:35.384 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:35 smithi149 ceph-mon[33409]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' 2024-05-03T17:50:35.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:35 smithi192 ceph-mon[33753]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:50:35.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:35 smithi192 ceph-mon[33753]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T17:50:35.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:35 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:35.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:35 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:35.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:35 smithi192 ceph-mon[33753]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' 2024-05-03T17:50:35.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:35 smithi122 ceph-mon[29744]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-05-03T17:50:35.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:35 smithi122 ceph-mon[29744]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T17:50:35.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:35 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:35.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:35 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:35.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:35 smithi122 ceph-mon[29744]: from='osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421]' entity='osd.9' 2024-05-03T17:50:36.128 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T17:50:36.128 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T17:50:36.128 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-03T17:50:36.128 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-03T17:50:36.129 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113364 s, 925 MB/s 2024-05-03T17:50:36.129 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-03T17:50:36.369 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:36 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:50:36.369 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:36 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:50:36.369 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:36 smithi122 ceph-mon[29744]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:36.369 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:36 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:36.370 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:36 smithi122 ceph-mon[29744]: osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421] boot 2024-05-03T17:50:36.370 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:36 smithi122 ceph-mon[29744]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T17:50:36.370 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:36 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:36.431 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:36 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:50:36.431 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:36 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:50:36.431 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:36 smithi149 ceph-mon[33409]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:36.431 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:36 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:36.431 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:36 smithi149 ceph-mon[33409]: osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421] boot 2024-05-03T17:50:36.432 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:36 smithi149 ceph-mon[33409]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T17:50:36.432 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:36 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:36.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:36 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:50:36.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:36 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:50:36.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:36 smithi192 ceph-mon[33753]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-03T17:50:36.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:36 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:36.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:36 smithi192 ceph-mon[33753]: osd.9 [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421] boot 2024-05-03T17:50:36.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:36 smithi192 ceph-mon[33753]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T17:50:36.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:36 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T17:50:36.747 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2024-05-03T17:50:36.993 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T17:50:37.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:37.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T17:50:37.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi122 2024-05-03T17:50:37.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:37.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:37.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:37.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:37.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:37.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:37.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi122 to 4272M 2024-05-03T17:50:37.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:37.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:37.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:37.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:37 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T17:50:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi122 2024-05-03T17:50:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:38.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:38.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi122 to 4272M 2024-05-03T17:50:38.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:38.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:38.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:38.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:37 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi122 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi122 to 4272M 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:38.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:38.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:37 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:38.749 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.752+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/635076360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8a4104920 msgr2=0x7fa8a4104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:38.749 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.752+0000 7fa8aa3ed700 1 --2- 172.21.15.149:0/635076360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8a4104920 0x7fa8a4104d80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa8980099c0 tx=0x7fa898009cd0 comp rx=0 tx=0).stop 2024-05-03T17:50:38.749 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.752+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/635076360 shutdown_connections 2024-05-03T17:50:38.749 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.752+0000 7fa8aa3ed700 1 --2- 172.21.15.149:0/635076360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8a41052c0 0x7fa8a410bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:38.749 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.752+0000 7fa8aa3ed700 1 --2- 172.21.15.149:0/635076360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a41018a0 0x7fa8a4101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.752+0000 7fa8aa3ed700 1 --2- 172.21.15.149:0/635076360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8a4104920 0x7fa8a4104d80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.752+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/635076360 >> 172.21.15.149:0/635076360 conn(0x7fa8a40fd1e0 msgr2=0x7fa8a40ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.752+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/635076360 shutdown_connections 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.752+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/635076360 wait complete. 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.752+0000 7fa8aa3ed700 1 Processor -- start 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.752+0000 7fa8aa3ed700 1 -- start start 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8aa3ed700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8a41018a0 0x7fa8a4199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8aa3ed700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8a4104920 0x7fa8a4199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8aa3ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a41052c0 0x7fa8a41a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8aa3ed700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a410f030 con 0x7fa8a4104920 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8aa3ed700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a410eeb0 con 0x7fa8a41018a0 2024-05-03T17:50:38.750 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8aa3ed700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a410f1b0 con 0x7fa8a41052c0 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8a41018a0 0x7fa8a4199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8a41018a0 0x7fa8a4199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:48552/0 (socket says 172.21.15.149:48552) 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a3fff700 1 -- 172.21.15.149:0/129269265 learned_addr learned my addr 172.21.15.149:0/129269265 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a898a700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a41052c0 0x7fa8a41a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a3fff700 1 -- 172.21.15.149:0/129269265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a41052c0 msgr2=0x7fa8a41a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a3fff700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a41052c0 0x7fa8a41a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a3fff700 1 -- 172.21.15.149:0/129269265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8a4104920 msgr2=0x7fa8a4199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a3fff700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8a4104920 0x7fa8a4199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a3fff700 1 -- 172.21.15.149:0/129269265 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a41a12d0 con 0x7fa8a41018a0 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a3fff700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8a41018a0 0x7fa8a4199670 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa89400d980 tx=0x7fa89400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a17fa700 1 -- 172.21.15.149:0/129269265 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894017030 con 0x7fa8a41018a0 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/129269265 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a41a15c0 con 0x7fa8a41018a0 2024-05-03T17:50:38.751 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a17fa700 1 -- 172.21.15.149:0/129269265 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8940044d0 con 0x7fa8a41018a0 2024-05-03T17:50:38.752 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8a17fa700 1 -- 172.21.15.149:0/129269265 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894003aa0 con 0x7fa8a41018a0 2024-05-03T17:50:38.752 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.753+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/129269265 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a41a1b50 con 0x7fa8a41018a0 2024-05-03T17:50:38.752 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.754+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/129269265 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a406a190 con 0x7fa8a41018a0 2024-05-03T17:50:38.752 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.756+0000 7fa8a17fa700 1 -- 172.21.15.149:0/129269265 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fa89401a070 con 0x7fa8a41018a0 2024-05-03T17:50:38.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.756+0000 7fa8a17fa700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa88c0b2880 0x7fa88c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:50:38.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.756+0000 7fa8a17fa700 1 -- 172.21.15.149:0/129269265 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7fa8940e1480 con 0x7fa8a41018a0 2024-05-03T17:50:38.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.757+0000 7fa8a37fe700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa88c0b2880 0x7fa88c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:50:38.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.757+0000 7fa8a37fe700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa88c0b2880 0x7fa88c0b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa898003790 tx=0x7fa898000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:50:38.755 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.761+0000 7fa8a17fa700 1 -- 172.21.15.149:0/129269265 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa8940a99a0 con 0x7fa8a41018a0 2024-05-03T17:50:38.965 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:38.970+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/129269265 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa8a40638e0 con 0x7fa88c0b2880 2024-05-03T17:50:39.836 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:39 smithi149 ceph-mon[33409]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:39.836 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:39 smithi149 ceph-mon[33409]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:50:39.837 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:39 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:50:39.837 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:39 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:50:39.837 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:39 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:39.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:39 smithi122 ceph-mon[29744]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:39.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:39 smithi122 ceph-mon[29744]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:50:39.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:39 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:50:39.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:39 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:50:39.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:39 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:40.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:39 smithi192 ceph-mon[33753]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:40.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:39 smithi192 ceph-mon[33753]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:50:40.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:39 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:50:40.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:39 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:50:40.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:39 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:41.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:41 smithi122 ceph-mon[29744]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:42.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:41 smithi149 ceph-mon[33409]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:42.078 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:41 smithi192 ceph-mon[33753]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:42.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:42 smithi122 ceph-mon[29744]: from='client.? 172.21.15.149:0/1722485439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af88e50f-98f1-460d-856b-a920dd2ae514"}]: dispatch 2024-05-03T17:50:42.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:42 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af88e50f-98f1-460d-856b-a920dd2ae514"}]: dispatch 2024-05-03T17:50:42.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:42 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af88e50f-98f1-460d-856b-a920dd2ae514"}]': finished 2024-05-03T17:50:42.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:42 smithi122 ceph-mon[29744]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T17:50:42.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:50:42.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:42 smithi122 ceph-mon[29744]: from='client.? 172.21.15.149:0/3173379930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:50:43.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:42 smithi149 ceph-mon[33409]: from='client.? 172.21.15.149:0/1722485439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af88e50f-98f1-460d-856b-a920dd2ae514"}]: dispatch 2024-05-03T17:50:43.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:42 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af88e50f-98f1-460d-856b-a920dd2ae514"}]: dispatch 2024-05-03T17:50:43.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:42 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af88e50f-98f1-460d-856b-a920dd2ae514"}]': finished 2024-05-03T17:50:43.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:42 smithi149 ceph-mon[33409]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T17:50:43.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:50:43.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:42 smithi149 ceph-mon[33409]: from='client.? 172.21.15.149:0/3173379930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:50:43.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:42 smithi192 ceph-mon[33753]: from='client.? 172.21.15.149:0/1722485439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af88e50f-98f1-460d-856b-a920dd2ae514"}]: dispatch 2024-05-03T17:50:43.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:42 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af88e50f-98f1-460d-856b-a920dd2ae514"}]: dispatch 2024-05-03T17:50:43.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:42 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af88e50f-98f1-460d-856b-a920dd2ae514"}]': finished 2024-05-03T17:50:43.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:42 smithi192 ceph-mon[33753]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T17:50:43.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:50:43.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:42 smithi192 ceph-mon[33753]: from='client.? 172.21.15.149:0/3173379930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:50:44.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:43 smithi192 ceph-mon[33753]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:44.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:43 smithi122 ceph-mon[29744]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:44.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:43 smithi149 ceph-mon[33409]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:46.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:45 smithi192 ceph-mon[33753]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:46.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:45 smithi122 ceph-mon[29744]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:46.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:45 smithi149 ceph-mon[33409]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:48.077 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:47 smithi149 ceph-mon[33409]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:48.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:47 smithi192 ceph-mon[33753]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:48.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:47 smithi122 ceph-mon[29744]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:50.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:49 smithi192 ceph-mon[33753]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:50.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T17:50:50.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:50.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:49 smithi192 ceph-mon[33753]: Deploying daemon osd.10 on smithi149 2024-05-03T17:50:50.155 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:49 smithi149 ceph-mon[33409]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:50.155 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T17:50:50.155 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:50.155 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:49 smithi149 ceph-mon[33409]: Deploying daemon osd.10 on smithi149 2024-05-03T17:50:50.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:49 smithi122 ceph-mon[29744]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:50.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T17:50:50.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:50.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:49 smithi122 ceph-mon[29744]: Deploying daemon osd.10 on smithi149 2024-05-03T17:50:52.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:51 smithi149 ceph-mon[33409]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:52.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:51 smithi192 ceph-mon[33753]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:52.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:51 smithi122 ceph-mon[29744]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:54.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:53 smithi192 ceph-mon[33753]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:54.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:53 smithi122 ceph-mon[29744]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:54.212 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:53 smithi149 ceph-mon[33409]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:56.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:56 smithi149 ceph-mon[33409]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:56.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:56.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:56.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:56.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:56 smithi192 ceph-mon[33753]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:56.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:56.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:56.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:56.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:56 smithi122 ceph-mon[29744]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:56.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:56.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:56.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:58.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:58 smithi149 ceph-mon[33409]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:58.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:58 smithi192 ceph-mon[33753]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:58.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:58 smithi122 ceph-mon[29744]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:50:59.132 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.136+0000 7fa8a17fa700 1 -- 172.21.15.149:0/129269265 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa8a40638e0 con 0x7fa88c0b2880 2024-05-03T17:50:59.133 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 10 on host 'smithi149' 2024-05-03T17:50:59.135 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/129269265 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa88c0b2880 msgr2=0x7fa88c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:59.135 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa88c0b2880 0x7fa88c0b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa898003790 tx=0x7fa898000bc0 comp rx=0 tx=0).stop 2024-05-03T17:50:59.135 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/129269265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8a41018a0 msgr2=0x7fa8a4199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:50:59.135 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8a41018a0 0x7fa8a4199670 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa89400d980 tx=0x7fa89400dd40 comp rx=0 tx=0).stop 2024-05-03T17:50:59.135 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/129269265 shutdown_connections 2024-05-03T17:50:59.135 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa88c0b2880 0x7fa88c0b4d40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:59.135 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a41052c0 0x7fa8a41a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:59.135 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8a41018a0 0x7fa8a4199670 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:59.136 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 --2- 172.21.15.149:0/129269265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8a4104920 0x7fa8a4199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:50:59.136 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/129269265 >> 172.21.15.149:0/129269265 conn(0x7fa8a40fd1e0 msgr2=0x7fa8a4107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:50:59.136 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/129269265 shutdown_connections 2024-05-03T17:50:59.136 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T17:50:59.139+0000 7fa8aa3ed700 1 -- 172.21.15.149:0/129269265 wait complete. 2024-05-03T17:50:59.507 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.507 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.507 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:59.507 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:59.507 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.507 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:59 smithi149 ceph-mon[33409]: from='osd.10 [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T17:50:59.508 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:59 smithi149 ceph-mon[33409]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T17:50:59.508 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.508 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.508 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:50:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:59.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:59.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:59.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:59 smithi192 ceph-mon[33753]: from='osd.10 [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T17:50:59.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:59 smithi192 ceph-mon[33753]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T17:50:59.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:50:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:59.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:50:59.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:50:59.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:59 smithi122 ceph-mon[29744]: from='osd.10 [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T17:50:59.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:59 smithi122 ceph-mon[29744]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T17:50:59.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:50:59.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:50:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:50:59.849 DEBUG:teuthology.orchestra.run.smithi149:osd.10> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.10.service 2024-05-03T17:50:59.854 INFO:tasks.cephadm:Deploying osd.11 on smithi192 with /dev/vg_nvme/lv_1... 2024-05-03T17:50:59.854 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T17:51:00.102 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:51:00.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:00 smithi149 ceph-mon[33409]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:51:00.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:00 smithi149 ceph-mon[33409]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T17:51:00.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:00 smithi149 ceph-mon[33409]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T17:51:00.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:00.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:00 smithi149 ceph-mon[33409]: from='osd.10 [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:51:00.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:00 smithi149 ceph-mon[33409]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:51:00.538 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 17:51:00 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T17:51:00.315+0000 7f3efaa35700 -1 osd.10 0 waiting for initial osdmap 2024-05-03T17:51:00.538 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 17:51:00 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T17:51:00.326+0000 7f3ef6828700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:51:00.553 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:00 smithi192 ceph-mon[33753]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:51:00.553 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:00 smithi192 ceph-mon[33753]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T17:51:00.553 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:00 smithi192 ceph-mon[33753]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T17:51:00.553 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:00.554 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:00 smithi192 ceph-mon[33753]: from='osd.10 [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:51:00.554 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:00 smithi192 ceph-mon[33753]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:51:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:00 smithi122 ceph-mon[29744]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:51:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:00 smithi122 ceph-mon[29744]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T17:51:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:00 smithi122 ceph-mon[29744]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T17:51:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:00 smithi122 ceph-mon[29744]: from='osd.10 [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:51:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:00 smithi122 ceph-mon[29744]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T17:51:01.505 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:01 smithi192 ceph-mon[33753]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:51:01.505 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:01 smithi192 ceph-mon[33753]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T17:51:01.505 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:01.505 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:01.586 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:01 smithi149 ceph-mon[33409]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:51:01.587 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:01 smithi149 ceph-mon[33409]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T17:51:01.587 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:01 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:01.587 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:01 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:01.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:01 smithi122 ceph-mon[29744]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T17:51:01.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:01 smithi122 ceph-mon[29744]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T17:51:01.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:01 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:01.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:01 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:02.191 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T17:51:02.192 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T17:51:02.192 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-05-03T17:51:02.192 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-05-03T17:51:02.192 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124769 s, 840 MB/s 2024-05-03T17:51:02.192 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-05-03T17:51:02.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:02 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:51:02.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:02 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:51:02.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:02 smithi149 ceph-mon[33409]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:51:02.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:02 smithi149 ceph-mon[33409]: osd.10 [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137] boot 2024-05-03T17:51:02.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:02 smithi149 ceph-mon[33409]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T17:51:02.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:02 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:02.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:02 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:51:02.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:02 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:51:02.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:02 smithi192 ceph-mon[33753]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:51:02.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:02 smithi192 ceph-mon[33753]: osd.10 [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137] boot 2024-05-03T17:51:02.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:02 smithi192 ceph-mon[33753]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T17:51:02.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:02 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:02.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:02 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:51:02.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:02 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:51:02.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:02 smithi122 ceph-mon[29744]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-03T17:51:02.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:02 smithi122 ceph-mon[29744]: osd.10 [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137] boot 2024-05-03T17:51:02.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:02 smithi122 ceph-mon[29744]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T17:51:02.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:02 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T17:51:02.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_1 2024-05-03T17:51:03.142 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:51:03.604 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:03 smithi192 ceph-mon[33753]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T17:51:03.604 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:03.604 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:03.604 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.604 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.604 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.605 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.605 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:03.605 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:03.605 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:03.605 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:03.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:03 smithi122 ceph-mon[29744]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T17:51:03.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:03.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:03.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:03.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:03.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:03.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:03 smithi149 ceph-mon[33409]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T17:51:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:03.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:03.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:03.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:03.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:04.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:04 smithi192 ceph-mon[33753]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:04.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:04 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi149 2024-05-03T17:51:04.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:04 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi149 to 1454M 2024-05-03T17:51:04.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:04 smithi122 ceph-mon[29744]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:04.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:04 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi149 2024-05-03T17:51:04.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:04 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi149 to 1454M 2024-05-03T17:51:04.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:04 smithi149 ceph-mon[33409]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:04.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:04 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi149 2024-05-03T17:51:04.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:04 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi149 to 1454M 2024-05-03T17:51:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.879+0000 7fd450c6a700 1 -- 172.21.15.192:0/1557141688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd44c1035f0 msgr2=0x7fd44c103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.879+0000 7fd450c6a700 1 --2- 172.21.15.192:0/1557141688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd44c1035f0 0x7fd44c103a50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009990 tx=0x7fd43c009ca0 comp rx=0 tx=0).stop 2024-05-03T17:51:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.879+0000 7fd450c6a700 1 -- 172.21.15.192:0/1557141688 shutdown_connections 2024-05-03T17:51:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.879+0000 7fd450c6a700 1 --2- 172.21.15.192:0/1557141688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd44c103f90 0x7fd44c10a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:04.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.879+0000 7fd450c6a700 1 --2- 172.21.15.192:0/1557141688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd44c1023f0 0x7fd44c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:04.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.879+0000 7fd450c6a700 1 --2- 172.21.15.192:0/1557141688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd44c1035f0 0x7fd44c103a50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:04.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.879+0000 7fd450c6a700 1 -- 172.21.15.192:0/1557141688 >> 172.21.15.192:0/1557141688 conn(0x7fd44c0fd990 msgr2=0x7fd44c0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:04.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.879+0000 7fd450c6a700 1 -- 172.21.15.192:0/1557141688 shutdown_connections 2024-05-03T17:51:04.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.879+0000 7fd450c6a700 1 -- 172.21.15.192:0/1557141688 wait complete. 2024-05-03T17:51:04.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.880+0000 7fd450c6a700 1 Processor -- start 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.880+0000 7fd450c6a700 1 -- start start 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.880+0000 7fd450c6a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd44c1023f0 0x7fd44c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.880+0000 7fd450c6a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd44c1035f0 0x7fd44c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.880+0000 7fd450c6a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd44c103f90 0x7fd44c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.880+0000 7fd450c6a700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c10df80 con 0x7fd44c1035f0 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.880+0000 7fd450c6a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c10de00 con 0x7fd44c103f90 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.880+0000 7fd450c6a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c10e100 con 0x7fd44c1023f0 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.880+0000 7fd44a59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd44c1023f0 0x7fd44c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.880+0000 7fd44a59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd44c1023f0 0x7fd44c198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57886/0 (socket says 172.21.15.192:57886) 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.880+0000 7fd44a59c700 1 -- 172.21.15.192:0/985793329 learned_addr learned my addr 172.21.15.192:0/985793329 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.881+0000 7fd44a59c700 1 -- 172.21.15.192:0/985793329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd44c103f90 msgr2=0x7fd44c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.881+0000 7fd44a59c700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd44c103f90 0x7fd44c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:04.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.881+0000 7fd44a59c700 1 -- 172.21.15.192:0/985793329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd44c1035f0 msgr2=0x7fd44c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:04.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.881+0000 7fd44a59c700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd44c1035f0 0x7fd44c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:04.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.881+0000 7fd44a59c700 1 -- 172.21.15.192:0/985793329 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c1a0290 con 0x7fd44c1023f0 2024-05-03T17:51:04.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.881+0000 7fd44a59c700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd44c1023f0 0x7fd44c198630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd44000b530 tx=0x7fd44000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:04.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.881+0000 7fd43b7fe700 1 -- 172.21.15.192:0/985793329 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440015020 con 0x7fd44c1023f0 2024-05-03T17:51:04.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.881+0000 7fd43b7fe700 1 -- 172.21.15.192:0/985793329 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd440011ce0 con 0x7fd44c1023f0 2024-05-03T17:51:04.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.881+0000 7fd450c6a700 1 -- 172.21.15.192:0/985793329 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44c1a0580 con 0x7fd44c1023f0 2024-05-03T17:51:04.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.881+0000 7fd43b7fe700 1 -- 172.21.15.192:0/985793329 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440010d60 con 0x7fd44c1023f0 2024-05-03T17:51:04.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.881+0000 7fd450c6a700 1 -- 172.21.15.192:0/985793329 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44c1a0b40 con 0x7fd44c1023f0 2024-05-03T17:51:04.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.882+0000 7fd450c6a700 1 -- 172.21.15.192:0/985793329 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd44c06a190 con 0x7fd44c1023f0 2024-05-03T17:51:04.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.883+0000 7fd43b7fe700 1 -- 172.21.15.192:0/985793329 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fd440018020 con 0x7fd44c1023f0 2024-05-03T17:51:04.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.884+0000 7fd43b7fe700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd4340b2900 0x7fd4340b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:04.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.884+0000 7fd43b7fe700 1 -- 172.21.15.192:0/985793329 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6694+0+0 (secure 0 0 0) 0x7fd4400e18f0 con 0x7fd44c1023f0 2024-05-03T17:51:04.880 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.884+0000 7fd449d9b700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd4340b2900 0x7fd4340b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:04.880 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.885+0000 7fd449d9b700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd4340b2900 0x7fd4340b4dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00b5c0 tx=0x7fd43c003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:04.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:04.888+0000 7fd43b7fe700 1 -- 172.21.15.192:0/985793329 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd44001f020 con 0x7fd44c1023f0 2024-05-03T17:51:05.095 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:05.099+0000 7fd450c6a700 1 -- 172.21.15.192:0/985793329 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd44c0638e0 con 0x7fd4340b2900 2024-05-03T17:51:05.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:05 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:51:05.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:05 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:51:05.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:05 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:05.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:05 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:51:05.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:05 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:51:05.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:05 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:05.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:05 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T17:51:05.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:05 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T17:51:05.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:05 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:06.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:06 smithi192 ceph-mon[33753]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-03T17:51:06.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:06 smithi192 ceph-mon[33753]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:51:06.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:06 smithi122 ceph-mon[29744]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-03T17:51:06.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:06 smithi122 ceph-mon[29744]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:51:06.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:06 smithi149 ceph-mon[33409]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-05-03T17:51:06.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:06 smithi149 ceph-mon[33409]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:51:08.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:08 smithi149 ceph-mon[33409]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-03T17:51:08.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:08 smithi149 ceph-mon[33409]: from='client.? 172.21.15.192:0/604724986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b54c1b4d-cdb5-42d9-979a-47f9a870b5cd"}]: dispatch 2024-05-03T17:51:08.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:08 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b54c1b4d-cdb5-42d9-979a-47f9a870b5cd"}]: dispatch 2024-05-03T17:51:08.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:08 smithi149 ceph-mon[33409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b54c1b4d-cdb5-42d9-979a-47f9a870b5cd"}]': finished 2024-05-03T17:51:08.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:08 smithi149 ceph-mon[33409]: osdmap e66: 12 total, 11 up, 12 in 2024-05-03T17:51:08.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:08 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:08.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:08 smithi192 ceph-mon[33753]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-03T17:51:08.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:08 smithi192 ceph-mon[33753]: from='client.? 172.21.15.192:0/604724986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b54c1b4d-cdb5-42d9-979a-47f9a870b5cd"}]: dispatch 2024-05-03T17:51:08.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:08 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b54c1b4d-cdb5-42d9-979a-47f9a870b5cd"}]: dispatch 2024-05-03T17:51:08.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:08 smithi192 ceph-mon[33753]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b54c1b4d-cdb5-42d9-979a-47f9a870b5cd"}]': finished 2024-05-03T17:51:08.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:08 smithi192 ceph-mon[33753]: osdmap e66: 12 total, 11 up, 12 in 2024-05-03T17:51:08.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:08 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:08.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:08 smithi122 ceph-mon[29744]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-03T17:51:08.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:08 smithi122 ceph-mon[29744]: from='client.? 172.21.15.192:0/604724986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b54c1b4d-cdb5-42d9-979a-47f9a870b5cd"}]: dispatch 2024-05-03T17:51:08.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:08 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b54c1b4d-cdb5-42d9-979a-47f9a870b5cd"}]: dispatch 2024-05-03T17:51:08.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:08 smithi122 ceph-mon[29744]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b54c1b4d-cdb5-42d9-979a-47f9a870b5cd"}]': finished 2024-05-03T17:51:08.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:08 smithi122 ceph-mon[29744]: osdmap e66: 12 total, 11 up, 12 in 2024-05-03T17:51:08.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:08 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:09.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:09 smithi192 ceph-mon[33753]: from='client.? 172.21.15.192:0/3839668681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:51:09.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:09 smithi122 ceph-mon[29744]: from='client.? 172.21.15.192:0/3839668681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:51:09.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:09 smithi149 ceph-mon[33409]: from='client.? 172.21.15.192:0/3839668681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T17:51:10.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:10 smithi192 ceph-mon[33753]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-03T17:51:10.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:10 smithi122 ceph-mon[29744]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-03T17:51:10.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:10 smithi149 ceph-mon[33409]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-03T17:51:12.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:12 smithi192 ceph-mon[33753]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-03T17:51:12.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:12 smithi122 ceph-mon[29744]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-03T17:51:12.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:12 smithi149 ceph-mon[33409]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-03T17:51:14.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:14 smithi192 ceph-mon[33753]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T17:51:14.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:14 smithi122 ceph-mon[29744]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T17:51:14.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:14 smithi149 ceph-mon[33409]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T17:51:16.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:16 smithi192 ceph-mon[33753]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:16.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T17:51:16.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:16 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:16.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:16 smithi192 ceph-mon[33753]: Deploying daemon osd.11 on smithi192 2024-05-03T17:51:16.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:16 smithi122 ceph-mon[29744]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:16.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T17:51:16.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:16 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:16.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:16 smithi122 ceph-mon[29744]: Deploying daemon osd.11 on smithi192 2024-05-03T17:51:16.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:16 smithi149 ceph-mon[33409]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:16.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T17:51:16.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:16 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:16.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:16 smithi149 ceph-mon[33409]: Deploying daemon osd.11 on smithi192 2024-05-03T17:51:18.596 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:18 smithi192 ceph-mon[33753]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:18.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:18 smithi122 ceph-mon[29744]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:18.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:18 smithi149 ceph-mon[33409]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:20.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:20 smithi192 ceph-mon[33753]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:20.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:20 smithi122 ceph-mon[29744]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:20.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:20 smithi149 ceph-mon[33409]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:22.514 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:22 smithi192 ceph-mon[33753]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:22.514 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:22 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:22.514 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:22 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:22.514 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:22 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:51:22.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:22 smithi122 ceph-mon[29744]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:22.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:22 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:22.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:22 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:22.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:22 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:51:22.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:22 smithi149 ceph-mon[33409]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:22.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:22 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:22.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:22 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:22.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:22 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:51:24.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:24 smithi122 ceph-mon[29744]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:24.748 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:24 smithi192 ceph-mon[33753]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:24.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:24 smithi149 ceph-mon[33409]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:25.612 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 11 on host 'smithi192' 2024-05-03T17:51:25.612 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.616+0000 7fd43b7fe700 1 -- 172.21.15.192:0/985793329 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd44c0638e0 con 0x7fd4340b2900 2024-05-03T17:51:25.618 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 -- 172.21.15.192:0/985793329 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd4340b2900 msgr2=0x7fd4340b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:25.618 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd4340b2900 0x7fd4340b4dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00b5c0 tx=0x7fd43c003860 comp rx=0 tx=0).stop 2024-05-03T17:51:25.618 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 -- 172.21.15.192:0/985793329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd44c1023f0 msgr2=0x7fd44c198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:25.619 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd44c1023f0 0x7fd44c198630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd44000b530 tx=0x7fd44000b8f0 comp rx=0 tx=0).stop 2024-05-03T17:51:25.619 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 -- 172.21.15.192:0/985793329 shutdown_connections 2024-05-03T17:51:25.619 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd4340b2900 0x7fd4340b4dc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:25.619 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd44c103f90 0x7fd44c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:25.619 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd44c1023f0 0x7fd44c198630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:25.619 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 --2- 172.21.15.192:0/985793329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd44c1035f0 0x7fd44c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:25.619 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 -- 172.21.15.192:0/985793329 >> 172.21.15.192:0/985793329 conn(0x7fd44c0fd990 msgr2=0x7fd44c106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:25.619 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 -- 172.21.15.192:0/985793329 shutdown_connections 2024-05-03T17:51:25.619 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:25.619+0000 7fd450c6a700 1 -- 172.21.15.192:0/985793329 wait complete. 2024-05-03T17:51:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:25 smithi192 ceph-mon[33753]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:25 smithi192 ceph-mon[33753]: from='osd.11 [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T17:51:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:25 smithi192 ceph-mon[33753]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T17:51:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:25 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:51:26.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:25 smithi122 ceph-mon[29744]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:26.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:26.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:26.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:25 smithi122 ceph-mon[29744]: from='osd.11 [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T17:51:26.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:25 smithi122 ceph-mon[29744]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T17:51:26.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:25 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:51:26.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:25 smithi149 ceph-mon[33409]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:26.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:26.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:26.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:25 smithi149 ceph-mon[33409]: from='osd.11 [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T17:51:26.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:25 smithi149 ceph-mon[33409]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T17:51:26.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:26.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:25 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:51:26.365 DEBUG:teuthology.orchestra.run.smithi192:osd.11> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.11.service 2024-05-03T17:51:26.368 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-03T17:51:26.368 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-03T17:51:26.617 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:51:27.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:27 smithi122 ceph-mon[29744]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T17:51:27.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:27 smithi122 ceph-mon[29744]: from='osd.11 [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:51:27.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:27 smithi122 ceph-mon[29744]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T17:51:27.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:27 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:27.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:27 smithi122 ceph-mon[29744]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:51:27.786 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:27 smithi192 ceph-mon[33753]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T17:51:27.786 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:27 smithi192 ceph-mon[33753]: from='osd.11 [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:51:27.786 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:27 smithi192 ceph-mon[33753]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T17:51:27.786 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:27 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:27.787 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:27 smithi192 ceph-mon[33753]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:51:27.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:27 smithi149 ceph-mon[33409]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T17:51:27.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:27 smithi149 ceph-mon[33409]: from='osd.11 [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:51:27.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:27 smithi149 ceph-mon[33409]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T17:51:27.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:27 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:27.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:27 smithi149 ceph-mon[33409]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-03T17:51:28.116 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 17:51:27 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T17:51:27.834+0000 7f90a4f8f700 -1 osd.11 0 waiting for initial osdmap 2024-05-03T17:51:28.116 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 17:51:27 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T17:51:27.848+0000 7f909cd7a700 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T17:51:28.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.399+0000 7f90c7152700 1 -- 172.21.15.122:0/3991583864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90c0103600 msgr2=0x7f90c0103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:28.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.399+0000 7f90c7152700 1 --2- 172.21.15.122:0/3991583864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90c0103600 0x7f90c0103a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0099c0 tx=0x7f90ac009cd0 comp rx=0 tx=0).stop 2024-05-03T17:51:28.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.399+0000 7f90c7152700 1 -- 172.21.15.122:0/3991583864 shutdown_connections 2024-05-03T17:51:28.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.399+0000 7f90c7152700 1 --2- 172.21.15.122:0/3991583864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0103fa0 0x7f90c010a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:28.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.399+0000 7f90c7152700 1 --2- 172.21.15.122:0/3991583864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c0102400 0x7f90c0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:28.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.399+0000 7f90c7152700 1 --2- 172.21.15.122:0/3991583864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90c0103600 0x7f90c0103a60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:28.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.399+0000 7f90c7152700 1 -- 172.21.15.122:0/3991583864 >> 172.21.15.122:0/3991583864 conn(0x7f90c00fd9a0 msgr2=0x7f90c00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:28.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.399+0000 7f90c7152700 1 -- 172.21.15.122:0/3991583864 shutdown_connections 2024-05-03T17:51:28.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.399+0000 7f90c7152700 1 -- 172.21.15.122:0/3991583864 wait complete. 2024-05-03T17:51:28.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c7152700 1 Processor -- start 2024-05-03T17:51:28.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c7152700 1 -- start start 2024-05-03T17:51:28.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c7152700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90c0102400 0x7f90c0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c7152700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0103600 0x7f90c0199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c7152700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c0103fa0 0x7f90c01a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c7152700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f90c010dea0 con 0x7f90c0102400 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c7152700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90c010dd20 con 0x7f90c0103600 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c7152700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f90c010e020 con 0x7f90c0103fa0 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c4eee700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90c0102400 0x7f90c0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c4eee700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90c0102400 0x7f90c0199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43174/0 (socket says 172.21.15.122:43174) 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c4eee700 1 -- 172.21.15.122:0/3299674202 learned_addr learned my addr 172.21.15.122:0/3299674202 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c56ef700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c0103fa0 0x7f90c01a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90bffff700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0103600 0x7f90c0199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c4eee700 1 -- 172.21.15.122:0/3299674202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c0103fa0 msgr2=0x7f90c01a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:28.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c4eee700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c0103fa0 0x7f90c01a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:28.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c4eee700 1 -- 172.21.15.122:0/3299674202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0103600 msgr2=0x7f90c0199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:28.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c4eee700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0103600 0x7f90c0199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:28.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.400+0000 7f90c4eee700 1 -- 172.21.15.122:0/3299674202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c0104820 con 0x7f90c0102400 2024-05-03T17:51:28.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.401+0000 7f90bffff700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0103600 0x7f90c0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:51:28.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.401+0000 7f90c4eee700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90c0102400 0x7f90c0199690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90b400e980 tx=0x7f90b400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:28.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.401+0000 7f90bdffb700 1 -- 172.21.15.122:0/3299674202 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b400cbd0 con 0x7f90c0102400 2024-05-03T17:51:28.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.401+0000 7f90c7152700 1 -- 172.21.15.122:0/3299674202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c0104b10 con 0x7f90c0102400 2024-05-03T17:51:28.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.401+0000 7f90bdffb700 1 -- 172.21.15.122:0/3299674202 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b40044e0 con 0x7f90c0102400 2024-05-03T17:51:28.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.401+0000 7f90bdffb700 1 -- 172.21.15.122:0/3299674202 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4003e00 con 0x7f90c0102400 2024-05-03T17:51:28.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.401+0000 7f90c7152700 1 -- 172.21.15.122:0/3299674202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c0105050 con 0x7f90c0102400 2024-05-03T17:51:28.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.402+0000 7f90c7152700 1 -- 172.21.15.122:0/3299674202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c006a190 con 0x7f90c0102400 2024-05-03T17:51:28.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.403+0000 7f90bdffb700 1 -- 172.21.15.122:0/3299674202 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f90b4003fa0 con 0x7f90c0102400 2024-05-03T17:51:28.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.404+0000 7f90bdffb700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f90b00b2880 0x7f90b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:28.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.404+0000 7f90bffff700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f90b00b2880 0x7f90b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:28.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.404+0000 7f90bdffb700 1 -- 172.21.15.122:0/3299674202 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6847+0+0 (secure 0 0 0) 0x7f90b40e20c0 con 0x7f90c0102400 2024-05-03T17:51:28.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.404+0000 7f90bffff700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f90b00b2880 0x7f90b00b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f90ac003820 tx=0x7f90ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:28.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.407+0000 7f90bdffb700 1 -- 172.21.15.122:0/3299674202 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f90b40aa3a0 con 0x7f90c0102400 2024-05-03T17:51:28.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:28 smithi122 ceph-mon[29744]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:28.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:28 smithi122 ceph-mon[29744]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:51:28.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:28 smithi122 ceph-mon[29744]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T17:51:28.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:28 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:28.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:28 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:28.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:28 smithi122 ceph-mon[29744]: from='osd.11 ' entity='osd.11' 2024-05-03T17:51:28.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:28 smithi192 ceph-mon[33753]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:28.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:28 smithi192 ceph-mon[33753]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:51:28.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:28 smithi192 ceph-mon[33753]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T17:51:28.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:28 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:28.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:28 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:28.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:28 smithi192 ceph-mon[33753]: from='osd.11 ' entity='osd.11' 2024-05-03T17:51:28.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.621+0000 7f90c7152700 1 -- 172.21.15.122:0/3299674202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f90c0105310 con 0x7f90c0102400 2024-05-03T17:51:28.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.622+0000 7f90bdffb700 1 -- 172.21.15.122:0/3299674202 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f90c0105310 con 0x7f90c0102400 2024-05-03T17:51:28.617 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:51:28.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 -- 172.21.15.122:0/3299674202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f90b00b2880 msgr2=0x7f90b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:28.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f90b00b2880 0x7f90b00b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f90ac003820 tx=0x7f90ac005fd0 comp rx=0 tx=0).stop 2024-05-03T17:51:28.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 -- 172.21.15.122:0/3299674202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90c0102400 msgr2=0x7f90c0199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:28.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90c0102400 0x7f90c0199690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90b400e980 tx=0x7f90b400ec90 comp rx=0 tx=0).stop 2024-05-03T17:51:28.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 -- 172.21.15.122:0/3299674202 shutdown_connections 2024-05-03T17:51:28.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f90b00b2880 0x7f90b00b4d40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:28.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c0103fa0 0x7f90c01a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:28.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90c0102400 0x7f90c0199690 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:28.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 --2- 172.21.15.122:0/3299674202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0103600 0x7f90c0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:28.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 -- 172.21.15.122:0/3299674202 >> 172.21.15.122:0/3299674202 conn(0x7f90c00fd9a0 msgr2=0x7f90c006b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:28.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 -- 172.21.15.122:0/3299674202 shutdown_connections 2024-05-03T17:51:28.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:28.624+0000 7f90c7152700 1 -- 172.21.15.122:0/3299674202 wait complete. 2024-05-03T17:51:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:28 smithi149 ceph-mon[33409]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:28 smithi149 ceph-mon[33409]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-03T17:51:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:28 smithi149 ceph-mon[33409]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T17:51:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:28 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:28 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:28 smithi149 ceph-mon[33409]: from='osd.11 ' entity='osd.11' 2024-05-03T17:51:29.376 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1714758661,"num_in_osds":12,"osd_in_since":1714758667,"num_remapped_pgs":0} 2024-05-03T17:51:29.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:29 smithi192 ceph-mon[33753]: purged_snaps scrub starts 2024-05-03T17:51:29.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:29 smithi192 ceph-mon[33753]: purged_snaps scrub ok 2024-05-03T17:51:29.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:29.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:29 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3299674202' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T17:51:29.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:29 smithi192 ceph-mon[33753]: osd.11 [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209] boot 2024-05-03T17:51:29.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:29 smithi192 ceph-mon[33753]: osdmap e69: 12 total, 12 up, 12 in 2024-05-03T17:51:29.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:29.683 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:29 smithi122 ceph-mon[29744]: purged_snaps scrub starts 2024-05-03T17:51:29.683 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:29 smithi122 ceph-mon[29744]: purged_snaps scrub ok 2024-05-03T17:51:29.683 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:29.683 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:29 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3299674202' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T17:51:29.683 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:29 smithi122 ceph-mon[29744]: osd.11 [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209] boot 2024-05-03T17:51:29.683 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:29 smithi122 ceph-mon[29744]: osdmap e69: 12 total, 12 up, 12 in 2024-05-03T17:51:29.683 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:29.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:29 smithi149 ceph-mon[33409]: purged_snaps scrub starts 2024-05-03T17:51:29.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:29 smithi149 ceph-mon[33409]: purged_snaps scrub ok 2024-05-03T17:51:29.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:29.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:29 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3299674202' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T17:51:29.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:29 smithi149 ceph-mon[33409]: osd.11 [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209] boot 2024-05-03T17:51:29.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:29 smithi149 ceph-mon[33409]: osdmap e69: 12 total, 12 up, 12 in 2024-05-03T17:51:29.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T17:51:30.377 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-03T17:51:30.616 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:51:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:30 smithi122 ceph-mon[29744]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:30.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:30 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:30.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:30 smithi149 ceph-mon[33409]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:30.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:30 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:30.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:30 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:30.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:30 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:30 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:30 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:30 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:30 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:30.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:30 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:30.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:30 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:30.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:30 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:30.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:30 smithi192 ceph-mon[33753]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-03T17:51:30.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:30 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:30.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:30 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:30.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:30 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:30 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:30 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:30 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T17:51:30.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:30 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:30.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:30 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:30.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:30 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:30.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:30 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:31.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:31 smithi122 ceph-mon[29744]: Detected new or changed devices on smithi192 2024-05-03T17:51:31.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:31 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi192 to 4272M 2024-05-03T17:51:31.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:31 smithi122 ceph-mon[29744]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T17:51:31.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:31 smithi149 ceph-mon[33409]: Detected new or changed devices on smithi192 2024-05-03T17:51:31.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:31 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi192 to 4272M 2024-05-03T17:51:31.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:31 smithi149 ceph-mon[33409]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T17:51:31.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:31 smithi192 ceph-mon[33753]: Detected new or changed devices on smithi192 2024-05-03T17:51:31.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:31 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi192 to 4272M 2024-05-03T17:51:31.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:31 smithi192 ceph-mon[33753]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T17:51:32.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.376+0000 7f1632649700 1 -- 172.21.15.122:0/1022905931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c1068f0 msgr2=0x7f162c108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:32.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.376+0000 7f1632649700 1 --2- 172.21.15.122:0/1022905931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c1068f0 0x7f162c108ce0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1620009990 tx=0x7f1620009ca0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.376+0000 7f1632649700 1 -- 172.21.15.122:0/1022905931 shutdown_connections 2024-05-03T17:51:32.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.376+0000 7f1632649700 1 --2- 172.21.15.122:0/1022905931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f162c109220 0x7f162c10b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.376+0000 7f1632649700 1 --2- 172.21.15.122:0/1022905931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f162c103fc0 0x7f162c1063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.376+0000 7f1632649700 1 --2- 172.21.15.122:0/1022905931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c1068f0 0x7f162c108ce0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.376+0000 7f1632649700 1 -- 172.21.15.122:0/1022905931 >> 172.21.15.122:0/1022905931 conn(0x7f162c0fd9f0 msgr2=0x7f162c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.376+0000 7f1632649700 1 -- 172.21.15.122:0/1022905931 shutdown_connections 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.376+0000 7f1632649700 1 -- 172.21.15.122:0/1022905931 wait complete. 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.376+0000 7f1632649700 1 Processor -- start 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.376+0000 7f1632649700 1 -- start start 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f1632649700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f162c103fc0 0x7f162c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f1632649700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c1068f0 0x7f162c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f1632649700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f162c109220 0x7f162c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f1632649700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10f050 con 0x7f162c1068f0 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f1632649700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10eed0 con 0x7f162c109220 2024-05-03T17:51:32.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f1632649700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10f1d0 con 0x7f162c103fc0 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f162b7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c1068f0 0x7f162c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f162bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f162c103fc0 0x7f162c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f162b7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c1068f0 0x7f162c199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49434/0 (socket says 172.21.15.122:49434) 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f162b7fe700 1 -- 172.21.15.122:0/647787297 learned_addr learned my addr 172.21.15.122:0/647787297 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f1630be6700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f162c109220 0x7f162c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f162b7fe700 1 -- 172.21.15.122:0/647787297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f162c103fc0 msgr2=0x7f162c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f162b7fe700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f162c103fc0 0x7f162c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f162b7fe700 1 -- 172.21.15.122:0/647787297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f162c109220 msgr2=0x7f162c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f162b7fe700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f162c109220 0x7f162c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f162b7fe700 1 -- 172.21.15.122:0/647787297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c1a12f0 con 0x7f162c1068f0 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f1630be6700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f162c109220 0x7f162c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.377+0000 7f162bfff700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f162c103fc0 0x7f162c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T17:51:32.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.378+0000 7f162b7fe700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c1068f0 0x7f162c199c70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f16200036f0 tx=0x7f16200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:32.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.378+0000 7f16297fa700 1 -- 172.21.15.122:0/647787297 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620027070 con 0x7f162c1068f0 2024-05-03T17:51:32.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.378+0000 7f1632649700 1 -- 172.21.15.122:0/647787297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c1a1580 con 0x7f162c1068f0 2024-05-03T17:51:32.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.378+0000 7f16297fa700 1 -- 172.21.15.122:0/647787297 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1620004080 con 0x7f162c1068f0 2024-05-03T17:51:32.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.378+0000 7f16297fa700 1 -- 172.21.15.122:0/647787297 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620022d50 con 0x7f162c1068f0 2024-05-03T17:51:32.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.378+0000 7f1632649700 1 -- 172.21.15.122:0/647787297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1a1ab0 con 0x7f162c1068f0 2024-05-03T17:51:32.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.379+0000 7f1632649700 1 -- 172.21.15.122:0/647787297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c06a190 con 0x7f162c1068f0 2024-05-03T17:51:32.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.381+0000 7f16297fa700 1 -- 172.21.15.122:0/647787297 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f162002a070 con 0x7f162c1068f0 2024-05-03T17:51:32.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.381+0000 7f16297fa700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f16140b2880 0x7f16140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:32.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.381+0000 7f162bfff700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f16140b2880 0x7f16140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:32.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.381+0000 7f16297fa700 1 -- 172.21.15.122:0/647787297 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f16200e3430 con 0x7f162c1068f0 2024-05-03T17:51:32.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.382+0000 7f162bfff700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f16140b2880 0x7f16140b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f161c0104e0 tx=0x7f161c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:32.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.385+0000 7f16297fa700 1 -- 172.21.15.122:0/647787297 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1620031070 con 0x7f162c1068f0 2024-05-03T17:51:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.590+0000 7f1632649700 1 -- 172.21.15.122:0/647787297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f162c02e510 con 0x7f162c1068f0 2024-05-03T17:51:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.590+0000 7f16297fa700 1 -- 172.21.15.122:0/647787297 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f1620013070 con 0x7f162c1068f0 2024-05-03T17:51:32.586 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:51:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.592+0000 7f1632649700 1 -- 172.21.15.122:0/647787297 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f16140b2880 msgr2=0x7f16140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.592+0000 7f1632649700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f16140b2880 0x7f16140b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f161c0104e0 tx=0x7f161c00b3c0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.592+0000 7f1632649700 1 -- 172.21.15.122:0/647787297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c1068f0 msgr2=0x7f162c199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.592+0000 7f1632649700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c1068f0 0x7f162c199c70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f16200036f0 tx=0x7f16200047b0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.592+0000 7f1632649700 1 -- 172.21.15.122:0/647787297 shutdown_connections 2024-05-03T17:51:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.592+0000 7f1632649700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f16140b2880 0x7f16140b4d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.592+0000 7f1632649700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f162c109220 0x7f162c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.592+0000 7f1632649700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f162c103fc0 0x7f162c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.592+0000 7f1632649700 1 --2- 172.21.15.122:0/647787297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c1068f0 0x7f162c199c70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.592+0000 7f1632649700 1 -- 172.21.15.122:0/647787297 >> 172.21.15.122:0/647787297 conn(0x7f162c0fd9f0 msgr2=0x7f162c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.593+0000 7f1632649700 1 -- 172.21.15.122:0/647787297 shutdown_connections 2024-05-03T17:51:32.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:32.593+0000 7f1632649700 1 -- 172.21.15.122:0/647787297 wait complete. 2024-05-03T17:51:32.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:32 smithi192 ceph-mon[33753]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:32.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:32 smithi122 ceph-mon[29744]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:32 smithi149 ceph-mon[33409]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:33.339 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1714758688,"num_in_osds":12,"osd_in_since":1714758667,"num_remapped_pgs":0} 2024-05-03T17:51:33.339 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T17:51:33.579 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:51:33.700 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:33 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/647787297' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T17:51:33.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:33 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/647787297' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T17:51:33.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:33 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/647787297' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T17:51:34.705 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:34 smithi122 ceph-mon[29744]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:34.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:34 smithi149 ceph-mon[33409]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:34.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:34 smithi192 ceph-mon[33753]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.210+0000 7f3959fa6700 1 -- 172.21.15.122:0/895793136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3954103750 msgr2=0x7f3954103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:35.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.210+0000 7f3959fa6700 1 --2- 172.21.15.122:0/895793136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3954103750 0x7f3954103b30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f39480099c0 tx=0x7f3948009cd0 comp rx=0 tx=0).stop 2024-05-03T17:51:35.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.210+0000 7f3959fa6700 1 -- 172.21.15.122:0/895793136 shutdown_connections 2024-05-03T17:51:35.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.210+0000 7f3959fa6700 1 --2- 172.21.15.122:0/895793136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3954108850 0x7f39541103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:35.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.210+0000 7f3959fa6700 1 --2- 172.21.15.122:0/895793136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3954103750 0x7f3954103b30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:35.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.210+0000 7f3959fa6700 1 --2- 172.21.15.122:0/895793136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3954104100 0x7f3954108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:35.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.210+0000 7f3959fa6700 1 -- 172.21.15.122:0/895793136 >> 172.21.15.122:0/895793136 conn(0x7f395407a600 msgr2=0x7f395407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:35.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.210+0000 7f3959fa6700 1 -- 172.21.15.122:0/895793136 shutdown_connections 2024-05-03T17:51:35.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.210+0000 7f3959fa6700 1 -- 172.21.15.122:0/895793136 wait complete. 2024-05-03T17:51:35.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f3959fa6700 1 Processor -- start 2024-05-03T17:51:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f3959fa6700 1 -- start start 2024-05-03T17:51:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f3959fa6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3954103750 0x7f395419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f3959fa6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3954104100 0x7f395419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f3959fa6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3954108850 0x7f39541998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f3959fa6700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3954113620 con 0x7f3954108850 2024-05-03T17:51:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f3959fa6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f39541134a0 con 0x7f3954103750 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f3959fa6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39541137a0 con 0x7f3954104100 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f3953fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3954108850 0x7f39541998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f3953fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3954108850 0x7f39541998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49452/0 (socket says 172.21.15.122:49452) 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f3953fff700 1 -- 172.21.15.122:0/349825444 learned_addr learned my addr 172.21.15.122:0/349825444 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.211+0000 7f39537fe700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3954103750 0x7f395419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3952ffd700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3954104100 0x7f395419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3953fff700 1 -- 172.21.15.122:0/349825444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3954104100 msgr2=0x7f395419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3953fff700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3954104100 0x7f395419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3953fff700 1 -- 172.21.15.122:0/349825444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3954103750 msgr2=0x7f395419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3953fff700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3954103750 0x7f395419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3953fff700 1 -- 172.21.15.122:0/349825444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395419a040 con 0x7f3954108850 2024-05-03T17:51:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3953fff700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3954108850 0x7f39541998e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f394c007ee0 tx=0x7f394c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:35.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3950ff9700 1 -- 172.21.15.122:0/349825444 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c00dbd0 con 0x7f3954108850 2024-05-03T17:51:35.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3950ff9700 1 -- 172.21.15.122:0/349825444 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394c00dd70 con 0x7f3954108850 2024-05-03T17:51:35.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3959fa6700 1 -- 172.21.15.122:0/349825444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395419a330 con 0x7f3954108850 2024-05-03T17:51:35.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3950ff9700 1 -- 172.21.15.122:0/349825444 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c005990 con 0x7f3954108850 2024-05-03T17:51:35.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.212+0000 7f3959fa6700 1 -- 172.21.15.122:0/349825444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39541a65f0 con 0x7f3954108850 2024-05-03T17:51:35.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.213+0000 7f3959fa6700 1 -- 172.21.15.122:0/349825444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f395410ba90 con 0x7f3954108850 2024-05-03T17:51:35.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.214+0000 7f3950ff9700 1 -- 172.21.15.122:0/349825444 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f394c01a020 con 0x7f3954108850 2024-05-03T17:51:35.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.215+0000 7f3950ff9700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f393c0b2880 0x7f393c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:35.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.215+0000 7f39537fe700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f393c0b2880 0x7f393c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:35.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.215+0000 7f3950ff9700 1 -- 172.21.15.122:0/349825444 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f394c0e1c20 con 0x7f3954108850 2024-05-03T17:51:35.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.215+0000 7f39537fe700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f393c0b2880 0x7f393c0b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f395419aa20 tx=0x7f3948005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:35.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.219+0000 7f3950ff9700 1 -- 172.21.15.122:0/349825444 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f394c010070 con 0x7f3954108850 2024-05-03T17:51:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.428+0000 7f3959fa6700 1 -- 172.21.15.122:0/349825444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f395410b470 con 0x7f3954108850 2024-05-03T17:51:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.429+0000 7f3950ff9700 1 -- 172.21.15.122:0/349825444 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18520 (secure 0 0 0) 0x7f394c0a9de0 con 0x7f3954108850 2024-05-03T17:51:35.425 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:51:35.425 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":70,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","created":"2024-05-03T17:44:17.415571+0000","modified":"2024-05-03T17:51:30.398612+0000","last_up_change":"2024-05-03T17:51:28.839117+0000","last_in_change":"2024-05-03T17:51:07.954070+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T17:47:40.539967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4b08628-0a29-44aa-b8b9-ed9657f12be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6803","nonce":1329364610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6805","nonce":1329364610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6809","nonce":1329364610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6807","nonce":1329364610}]},"public_addr":"172.21.15.122:6803/1329364610","cluster_addr":"172.21.15.122:6805/1329364610","heartbeat_back_addr":"172.21.15.122:6809/1329364610","heartbeat_front_addr":"172.21.15.122:6807/1329364610","state":["exists","up"]},{"osd":1,"uuid":"ba550787-ce76-4944-8ebd-97e19590bb3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6801","nonce":1402463302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6803","nonce":1402463302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6807","nonce":1402463302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6805","nonce":1402463302}]},"public_addr":"172.21.15.149:6801/1402463302","cluster_addr":"172.21.15.149:6803/1402463302","heartbeat_back_addr":"172.21.15.149:6807/1402463302","heartbeat_front_addr":"172.21.15.149:6805/1402463302","state":["exists","up"]},{"osd":2,"uuid":"35ae7c47-d710-47c5-b0ec-91fa12b5be07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6801","nonce":3757191110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6803","nonce":3757191110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6807","nonce":3757191110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6805","nonce":3757191110}]},"public_addr":"172.21.15.192:6801/3757191110","cluster_addr":"172.21.15.192:6803/3757191110","heartbeat_back_addr":"172.21.15.192:6807/3757191110","heartbeat_front_addr":"172.21.15.192:6805/3757191110","state":["exists","up"]},{"osd":3,"uuid":"9973c0c1-ae73-497a-845e-664a20666df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6811","nonce":2359143764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6813","nonce":2359143764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6817","nonce":2359143764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6815","nonce":2359143764}]},"public_addr":"172.21.15.122:6811/2359143764","cluster_addr":"172.21.15.122:6813/2359143764","heartbeat_back_addr":"172.21.15.122:6817/2359143764","heartbeat_front_addr":"172.21.15.122:6815/2359143764","state":["exists","up"]},{"osd":4,"uuid":"da71dc18-94b9-4f7d-a061-7cc7b77db282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3627037532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3627037532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3627037532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3627037532}]},"public_addr":"172.21.15.149:6809/3627037532","cluster_addr":"172.21.15.149:6811/3627037532","heartbeat_back_addr":"172.21.15.149:6815/3627037532","heartbeat_front_addr":"172.21.15.149:6813/3627037532","state":["exists","up"]},{"osd":5,"uuid":"bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6809","nonce":455779485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6811","nonce":455779485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6815","nonce":455779485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6813","nonce":455779485}]},"public_addr":"172.21.15.192:6809/455779485","cluster_addr":"172.21.15.192:6811/455779485","heartbeat_back_addr":"172.21.15.192:6815/455779485","heartbeat_front_addr":"172.21.15.192:6813/455779485","state":["exists","up"]},{"osd":6,"uuid":"4150bfe5-a0e4-4bae-914a-75598f017519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6819","nonce":3419421998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6821","nonce":3419421998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6825","nonce":3419421998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6823","nonce":3419421998}]},"public_addr":"172.21.15.122:6819/3419421998","cluster_addr":"172.21.15.122:6821/3419421998","heartbeat_back_addr":"172.21.15.122:6825/3419421998","heartbeat_front_addr":"172.21.15.122:6823/3419421998","state":["exists","up"]},{"osd":7,"uuid":"71e01812-e511-44a0-a143-2afca510ae8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6817","nonce":139747549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6819","nonce":139747549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6823","nonce":139747549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6821","nonce":139747549}]},"public_addr":"172.21.15.149:6817/139747549","cluster_addr":"172.21.15.149:6819/139747549","heartbeat_back_addr":"172.21.15.149:6823/139747549","heartbeat_front_addr":"172.21.15.149:6821/139747549","state":["exists","up"]},{"osd":8,"uuid":"5294516d-fac6-41d0-a635-6bc1d7eca585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6817","nonce":2080816744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6819","nonce":2080816744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6823","nonce":2080816744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6821","nonce":2080816744}]},"public_addr":"172.21.15.192:6817/2080816744","cluster_addr":"172.21.15.192:6819/2080816744","heartbeat_back_addr":"172.21.15.192:6823/2080816744","heartbeat_front_addr":"172.21.15.192:6821/2080816744","state":["exists","up"]},{"osd":9,"uuid":"f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6827","nonce":1773691421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6829","nonce":1773691421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6833","nonce":1773691421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6831","nonce":1773691421}]},"public_addr":"172.21.15.122:6827/1773691421","cluster_addr":"172.21.15.122:6829/1773691421","heartbeat_back_addr":"172.21.15.122:6833/1773691421","heartbeat_front_addr":"172.21.15.122:6831/1773691421","state":["exists","up"]},{"osd":10,"uuid":"af88e50f-98f1-460d-856b-a920dd2ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6825","nonce":1954186137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6827","nonce":1954186137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6831","nonce":1954186137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6829","nonce":1954186137}]},"public_addr":"172.21.15.149:6825/1954186137","cluster_addr":"172.21.15.149:6827/1954186137","heartbeat_back_addr":"172.21.15.149:6831/1954186137","heartbeat_front_addr":"172.21.15.149:6829/1954186137","state":["exists","up"]},{"osd":11,"uuid":"b54c1b4d-cdb5-42d9-979a-47f9a870b5cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6825","nonce":2098474209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6827","nonce":2098474209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6831","nonce":2098474209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6829","nonce":2098474209}]},"public_addr":"172.21.15.192:6825/2098474209","cluster_addr":"172.21.15.192:6827/2098474209","heartbeat_back_addr":"172.21.15.192:6831/2098474209","heartbeat_front_addr":"172.21.15.192:6829/2098474209","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:46:50.049769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:13.518485+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:37.473023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:01.809640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:26.648514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:51.268056+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:16.294834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:42.079065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:07.609271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:33.885127+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:59.982708+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:51:26.461369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/2089907395":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6801/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:0/2001860625":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/1888284988":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/256574472":"2024-05-04T17:44:52.244049+0000","172.21.15.122:6801/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/64546794":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/584846864":"2024-05-04T17:45:18.454406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T17:51:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 -- 172.21.15.122:0/349825444 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f393c0b2880 msgr2=0x7f393c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f393c0b2880 0x7f393c0b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f395419aa20 tx=0x7f3948005c40 comp rx=0 tx=0).stop 2024-05-03T17:51:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 -- 172.21.15.122:0/349825444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3954108850 msgr2=0x7f39541998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3954108850 0x7f39541998e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f394c007ee0 tx=0x7f394c00d3b0 comp rx=0 tx=0).stop 2024-05-03T17:51:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 -- 172.21.15.122:0/349825444 shutdown_connections 2024-05-03T17:51:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f393c0b2880 0x7f393c0b4d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3954108850 0x7f39541998e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3954103750 0x7f395419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 --2- 172.21.15.122:0/349825444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3954104100 0x7f395419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 -- 172.21.15.122:0/349825444 >> 172.21.15.122:0/349825444 conn(0x7f395407a600 msgr2=0x7f3954101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 -- 172.21.15.122:0/349825444 shutdown_connections 2024-05-03T17:51:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:35.432+0000 7f3959fa6700 1 -- 172.21.15.122:0/349825444 wait complete. 2024-05-03T17:51:36.176 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-03T17:47:40.539967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-03T17:51:36.177 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-03T17:51:36.412 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:51:36.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:36 smithi122 ceph-mon[29744]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:36.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:36 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/349825444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:51:36.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:36 smithi149 ceph-mon[33409]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:36.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:36 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/349825444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:51:36.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:36 smithi192 ceph-mon[33753]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:36.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:36 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/349825444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:51:38.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.073+0000 7f24d4258700 1 -- 172.21.15.122:0/2703075371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24cc103f20 msgr2=0x7f24cc110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.073+0000 7f24d4258700 1 --2- 172.21.15.122:0/2703075371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24cc103f20 0x7f24cc110310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f24c80099c0 tx=0x7f24c8009cd0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.073+0000 7f24d4258700 1 -- 172.21.15.122:0/2703075371 shutdown_connections 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.073+0000 7f24d4258700 1 --2- 172.21.15.122:0/2703075371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24cc103f20 0x7f24cc110310 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.073+0000 7f24d4258700 1 --2- 172.21.15.122:0/2703075371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24cc109540 0x7f24cc109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.073+0000 7f24d4258700 1 --2- 172.21.15.122:0/2703075371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24cc103560 0x7f24cc1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.073+0000 7f24d4258700 1 -- 172.21.15.122:0/2703075371 >> 172.21.15.122:0/2703075371 conn(0x7f24cc07a660 msgr2=0x7f24cc07aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.073+0000 7f24d4258700 1 -- 172.21.15.122:0/2703075371 shutdown_connections 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.073+0000 7f24d4258700 1 -- 172.21.15.122:0/2703075371 wait complete. 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d4258700 1 Processor -- start 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d4258700 1 -- start start 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d4258700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24cc103560 0x7f24cc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d4258700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24cc103f20 0x7f24cc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d4258700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24cc109540 0x7f24cc199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d4258700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc113650 con 0x7f24cc103560 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d4258700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1134d0 con 0x7f24cc109540 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d4258700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1137d0 con 0x7f24cc103f20 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d1ff4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24cc103560 0x7f24cc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d1ff4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24cc103560 0x7f24cc19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49478/0 (socket says 172.21.15.122:49478) 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d1ff4700 1 -- 172.21.15.122:0/1287400753 learned_addr learned my addr 172.21.15.122:0/1287400753 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d17f3700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24cc103f20 0x7f24cc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d27f5700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24cc109540 0x7f24cc199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d1ff4700 1 -- 172.21.15.122:0/1287400753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24cc103f20 msgr2=0x7f24cc19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d1ff4700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24cc103f20 0x7f24cc19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d1ff4700 1 -- 172.21.15.122:0/1287400753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24cc109540 msgr2=0x7f24cc199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d1ff4700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24cc109540 0x7f24cc199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d1ff4700 1 -- 172.21.15.122:0/1287400753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc19a090 con 0x7f24cc103560 2024-05-03T17:51:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d27f5700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24cc109540 0x7f24cc199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:51:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.074+0000 7f24d1ff4700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24cc103560 0x7f24cc19f6d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f24c4007f10 tx=0x7f24c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.075+0000 7f24c2ffd700 1 -- 172.21.15.122:0/1287400753 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c400ddd0 con 0x7f24cc103560 2024-05-03T17:51:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.075+0000 7f24d4258700 1 -- 172.21.15.122:0/1287400753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24cc19a380 con 0x7f24cc103560 2024-05-03T17:51:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.075+0000 7f24c2ffd700 1 -- 172.21.15.122:0/1287400753 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c4004bb0 con 0x7f24cc103560 2024-05-03T17:51:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.075+0000 7f24c2ffd700 1 -- 172.21.15.122:0/1287400753 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c40046a0 con 0x7f24cc103560 2024-05-03T17:51:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.075+0000 7f24d4258700 1 -- 172.21.15.122:0/1287400753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24cc1a65a0 con 0x7f24cc103560 2024-05-03T17:51:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.076+0000 7f24d4258700 1 -- 172.21.15.122:0/1287400753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24cc104bb0 con 0x7f24cc103560 2024-05-03T17:51:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.077+0000 7f24c2ffd700 1 -- 172.21.15.122:0/1287400753 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f24c401c020 con 0x7f24cc103560 2024-05-03T17:51:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.078+0000 7f24c2ffd700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f24b80b2880 0x7f24b80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.078+0000 7f24d17f3700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f24b80b2880 0x7f24b80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.078+0000 7f24c2ffd700 1 -- 172.21.15.122:0/1287400753 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f24c40e2030 con 0x7f24cc103560 2024-05-03T17:51:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.078+0000 7f24d17f3700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f24b80b2880 0x7f24b80b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f24cc07acd0 tx=0x7f24bc00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.081+0000 7f24c2ffd700 1 -- 172.21.15.122:0/1287400753 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f24c4023030 con 0x7f24cc103560 2024-05-03T17:51:38.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.284+0000 7f24d4258700 1 -- 172.21.15.122:0/1287400753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f24cc02e510 con 0x7f24cc103560 2024-05-03T17:51:38.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.284+0000 7f24c2ffd700 1 -- 172.21.15.122:0/1287400753 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f24c4010070 con 0x7f24cc103560 2024-05-03T17:51:38.280 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-05-03T17:51:38.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.286+0000 7f24d4258700 1 -- 172.21.15.122:0/1287400753 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f24b80b2880 msgr2=0x7f24b80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:38.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.286+0000 7f24d4258700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f24b80b2880 0x7f24b80b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f24cc07acd0 tx=0x7f24bc00b3c0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.286+0000 7f24d4258700 1 -- 172.21.15.122:0/1287400753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24cc103560 msgr2=0x7f24cc19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:38.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.286+0000 7f24d4258700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24cc103560 0x7f24cc19f6d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f24c4007f10 tx=0x7f24c400d3b0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.286+0000 7f24d4258700 1 -- 172.21.15.122:0/1287400753 shutdown_connections 2024-05-03T17:51:38.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.286+0000 7f24d4258700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f24b80b2880 0x7f24b80b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.286+0000 7f24d4258700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24cc109540 0x7f24cc199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.286+0000 7f24d4258700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24cc103560 0x7f24cc19f6d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.286+0000 7f24d4258700 1 --2- 172.21.15.122:0/1287400753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24cc103f20 0x7f24cc19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:38.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.286+0000 7f24d4258700 1 -- 172.21.15.122:0/1287400753 >> 172.21.15.122:0/1287400753 conn(0x7f24cc07a660 msgr2=0x7f24cc1021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:38.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.287+0000 7f24d4258700 1 -- 172.21.15.122:0/1287400753 shutdown_connections 2024-05-03T17:51:38.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:38.287+0000 7f24d4258700 1 -- 172.21.15.122:0/1287400753 wait complete. 2024-05-03T17:51:38.644 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:38 smithi122 ceph-mon[29744]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:38.644 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:38 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1287400753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T17:51:38.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:38 smithi149 ceph-mon[33409]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:38.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:38 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/1287400753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T17:51:38.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:38 smithi192 ceph-mon[33753]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:38.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:38 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/1287400753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T17:51:38.975 INFO:tasks.cephadm:Adding mds.a on smithi122 2024-05-03T17:51:38.975 INFO:tasks.cephadm:Adding mds.d on smithi122 2024-05-03T17:51:38.975 INFO:tasks.cephadm:Adding mds.g on smithi122 2024-05-03T17:51:38.975 INFO:tasks.cephadm:Adding mds.j on smithi122 2024-05-03T17:51:38.975 INFO:tasks.cephadm:Adding mds.b on smithi149 2024-05-03T17:51:38.975 INFO:tasks.cephadm:Adding mds.e on smithi149 2024-05-03T17:51:38.975 INFO:tasks.cephadm:Adding mds.h on smithi149 2024-05-03T17:51:38.976 INFO:tasks.cephadm:Adding mds.k on smithi149 2024-05-03T17:51:38.976 INFO:tasks.cephadm:Adding mds.c on smithi192 2024-05-03T17:51:38.976 INFO:tasks.cephadm:Adding mds.f on smithi192 2024-05-03T17:51:38.976 INFO:tasks.cephadm:Adding mds.i on smithi192 2024-05-03T17:51:38.976 INFO:tasks.cephadm:Adding mds.l on smithi192 2024-05-03T17:51:38.976 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l' 2024-05-03T17:51:39.236 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T17:51:40.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:40 smithi122 ceph-mon[29744]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:40.714 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:40 smithi192 ceph-mon[33753]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:40.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:40 smithi149 ceph-mon[33409]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:40.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.953+0000 7f9c2c794700 1 -- 172.21.15.192:0/2594101124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c24106960 msgr2=0x7f9c24108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:40.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.953+0000 7f9c2c794700 1 --2- 172.21.15.192:0/2594101124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c24106960 0x7f9c24108d50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00b0a0 tx=0x7f9c1c00b3b0 comp rx=0 tx=0).stop 2024-05-03T17:51:40.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.953+0000 7f9c2c794700 1 -- 172.21.15.192:0/2594101124 shutdown_connections 2024-05-03T17:51:40.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.953+0000 7f9c2c794700 1 --2- 172.21.15.192:0/2594101124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c24109290 0x7f9c2410b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:40.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.953+0000 7f9c2c794700 1 --2- 172.21.15.192:0/2594101124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c24104030 0x7f9c24106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:40.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.953+0000 7f9c2c794700 1 --2- 172.21.15.192:0/2594101124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c24106960 0x7f9c24108d50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.953+0000 7f9c2c794700 1 -- 172.21.15.192:0/2594101124 >> 172.21.15.192:0/2594101124 conn(0x7f9c240fd9a0 msgr2=0x7f9c240ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2c794700 1 -- 172.21.15.192:0/2594101124 shutdown_connections 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2c794700 1 -- 172.21.15.192:0/2594101124 wait complete. 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2c794700 1 Processor -- start 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2c794700 1 -- start start 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2c794700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c24104030 0x7f9c24199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2c794700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c24106960 0x7f9c24199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2c794700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c24109290 0x7f9c241a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2c794700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2410f080 con 0x7f9c24106960 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2c794700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2410ef00 con 0x7f9c24104030 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2c794700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2410f200 con 0x7f9c24109290 2024-05-03T17:51:40.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2ad31700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c24109290 0x7f9c241a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2ad31700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c24109290 0x7f9c241a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:55922/0 (socket says 172.21.15.192:55922) 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2ad31700 1 -- 172.21.15.192:0/2732539826 learned_addr learned my addr 172.21.15.192:0/2732539826 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.954+0000 7f9c2a530700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c24104030 0x7f9c24199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c29d2f700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c24106960 0x7f9c24199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c2ad31700 1 -- 172.21.15.192:0/2732539826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c24104030 msgr2=0x7f9c24199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c2ad31700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c24104030 0x7f9c24199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c2ad31700 1 -- 172.21.15.192:0/2732539826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c24106960 msgr2=0x7f9c24199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c2ad31700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c24106960 0x7f9c24199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c2ad31700 1 -- 172.21.15.192:0/2732539826 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c241a1470 con 0x7f9c24109290 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c2ad31700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c24109290 0x7f9c241a0dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400b4b0 tx=0x7f9c1400b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c1b7fe700 1 -- 172.21.15.192:0/2732539826 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14015020 con 0x7f9c24109290 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c1b7fe700 1 -- 172.21.15.192:0/2732539826 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c14011ce0 con 0x7f9c24109290 2024-05-03T17:51:40.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c2c794700 1 -- 172.21.15.192:0/2732539826 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c241a1760 con 0x7f9c24109290 2024-05-03T17:51:40.954 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c1b7fe700 1 -- 172.21.15.192:0/2732539826 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14010be0 con 0x7f9c24109290 2024-05-03T17:51:40.954 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.955+0000 7f9c2c794700 1 -- 172.21.15.192:0/2732539826 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c241a1d20 con 0x7f9c24109290 2024-05-03T17:51:40.954 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.956+0000 7f9c2c794700 1 -- 172.21.15.192:0/2732539826 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c2406a190 con 0x7f9c24109290 2024-05-03T17:51:40.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.958+0000 7f9c1b7fe700 1 -- 172.21.15.192:0/2732539826 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9c14018020 con 0x7f9c24109290 2024-05-03T17:51:40.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.959+0000 7f9c1b7fe700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9c100b2980 0x7f9c100b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:40.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.959+0000 7f9c1b7fe700 1 -- 172.21.15.192:0/2732539826 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f9c140e1780 con 0x7f9c24109290 2024-05-03T17:51:40.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.959+0000 7f9c2a530700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9c100b2980 0x7f9c100b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:40.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.960+0000 7f9c2a530700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9c100b2980 0x7f9c100b4e40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009a10 tx=0x7f9c20009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:40.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:40.962+0000 7f9c1b7fe700 1 -- 172.21.15.192:0/2732539826 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9c1401f030 con 0x7f9c24109290 2024-05-03T17:51:41.171 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.174+0000 7f9c2c794700 1 -- 172.21.15.192:0/2732539826 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}) -- 0x7f9c24063ae0 con 0x7f9c100b2980 2024-05-03T17:51:41.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.184+0000 7f9c1b7fe700 1 -- 172.21.15.192:0/2732539826 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9c24063ae0 con 0x7f9c100b2980 2024-05-03T17:51:41.179 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mds.all update... 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 -- 172.21.15.192:0/2732539826 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9c100b2980 msgr2=0x7f9c100b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9c100b2980 0x7f9c100b4e40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009a10 tx=0x7f9c20009190 comp rx=0 tx=0).stop 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 -- 172.21.15.192:0/2732539826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c24109290 msgr2=0x7f9c241a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c24109290 0x7f9c241a0dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400b4b0 tx=0x7f9c1400b7c0 comp rx=0 tx=0).stop 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 -- 172.21.15.192:0/2732539826 shutdown_connections 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9c100b2980 0x7f9c100b4e40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c24109290 0x7f9c241a0dd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c24104030 0x7f9c24199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 --2- 172.21.15.192:0/2732539826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c24106960 0x7f9c24199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 -- 172.21.15.192:0/2732539826 >> 172.21.15.192:0/2732539826 conn(0x7f9c240fd9a0 msgr2=0x7f9c24105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 -- 172.21.15.192:0/2732539826 shutdown_connections 2024-05-03T17:51:41.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-03T17:51:41.186+0000 7f9c2c794700 1 -- 172.21.15.192:0/2732539826 wait complete. 2024-05-03T17:51:41.828 DEBUG:teuthology.orchestra.run.smithi122:mds.a> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.a.service 2024-05-03T17:51:41.831 DEBUG:teuthology.orchestra.run.smithi122:mds.d> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.d.service 2024-05-03T17:51:41.874 DEBUG:teuthology.orchestra.run.smithi122:mds.g> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.g.service 2024-05-03T17:51:41.917 DEBUG:teuthology.orchestra.run.smithi122:mds.j> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.j.service 2024-05-03T17:51:41.960 DEBUG:teuthology.orchestra.run.smithi149:mds.b> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.b.service 2024-05-03T17:51:41.962 DEBUG:teuthology.orchestra.run.smithi149:mds.e> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.e.service 2024-05-03T17:51:42.005 DEBUG:teuthology.orchestra.run.smithi149:mds.h> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.h.service 2024-05-03T17:51:42.048 DEBUG:teuthology.orchestra.run.smithi149:mds.k> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.k.service 2024-05-03T17:51:42.091 DEBUG:teuthology.orchestra.run.smithi192:mds.c> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.c.service 2024-05-03T17:51:42.094 DEBUG:teuthology.orchestra.run.smithi192:mds.f> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.f.service 2024-05-03T17:51:42.137 DEBUG:teuthology.orchestra.run.smithi192:mds.i> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.i.service 2024-05-03T17:51:42.180 DEBUG:teuthology.orchestra.run.smithi192:mds.l> sudo journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.l.service 2024-05-03T17:51:42.223 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-03T17:51:42.223 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:51:42.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:42 smithi149 ceph-mon[33409]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:42.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:42 smithi149 ceph-mon[33409]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:51:42.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:42 smithi149 ceph-mon[33409]: Saving service mds.all spec with placement smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-05-03T17:51:42.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:42.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:51:42.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:42.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:42.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:42.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:42.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:42.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:42 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:42.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:42 smithi192 ceph-mon[33753]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:42.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:42 smithi192 ceph-mon[33753]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:51:42.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:42 smithi192 ceph-mon[33753]: Saving service mds.all spec with placement smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-05-03T17:51:42.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:42.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:51:42.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:42.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:42.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:42.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:42.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:42.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:42 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:42 smithi122 ceph-mon[29744]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:42 smithi122 ceph-mon[29744]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:51:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:42 smithi122 ceph-mon[29744]: Saving service mds.all spec with placement smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-05-03T17:51:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:51:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:51:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:42 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:42.467 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:51:43.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:43 smithi122 ceph-mon[29744]: Deploying daemon mds.l on smithi192 2024-05-03T17:51:43.519 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:43 smithi192 ceph-mon[33753]: Deploying daemon mds.l on smithi192 2024-05-03T17:51:43.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:43 smithi149 ceph-mon[33409]: Deploying daemon mds.l on smithi192 2024-05-03T17:51:44.116 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 17:51:44 smithi192 systemd[1]: Starting Ceph mds.l for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:51:44.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3286119614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4ac06c1a0 msgr2=0x7fb4ac06c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:44.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 --2- 172.21.15.122:0/3286119614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4ac06c1a0 0x7fb4ac06c600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb498009990 tx=0x7fb498009ca0 comp rx=0 tx=0).stop 2024-05-03T17:51:44.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3286119614 shutdown_connections 2024-05-03T17:51:44.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 --2- 172.21.15.122:0/3286119614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ac105ee0 0x7fb4ac106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:44.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 --2- 172.21.15.122:0/3286119614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ac1083d0 0x7fb4ac06bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:44.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 --2- 172.21.15.122:0/3286119614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4ac06c1a0 0x7fb4ac06c600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:44.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3286119614 >> 172.21.15.122:0/3286119614 conn(0x7fb4ac079350 msgr2=0x7fb4ac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:44.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3286119614 shutdown_connections 2024-05-03T17:51:44.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3286119614 wait complete. 2024-05-03T17:51:44.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 Processor -- start 2024-05-03T17:51:44.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 -- start start 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ac06c1a0 0x7fb4ac1a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4ac105ee0 0x7fb4ac1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ac1083d0 0x7fb4ac1a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac119c50 con 0x7fb4ac105ee0 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac119ad0 con 0x7fb4ac06c1a0 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.276+0000 7fb4b2d9e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac119dd0 con 0x7fb4ac1083d0 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4abfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4ac105ee0 0x7fb4ac1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4abfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4ac105ee0 0x7fb4ac1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36970/0 (socket says 172.21.15.122:36970) 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4abfff700 1 -- 172.21.15.122:0/3094679116 learned_addr learned my addr 172.21.15.122:0/3094679116 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4b133b700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ac1083d0 0x7fb4ac1a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4b0b3a700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ac06c1a0 0x7fb4ac1a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4abfff700 1 -- 172.21.15.122:0/3094679116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ac1083d0 msgr2=0x7fb4ac1a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4abfff700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ac1083d0 0x7fb4ac1a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:44.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4abfff700 1 -- 172.21.15.122:0/3094679116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ac06c1a0 msgr2=0x7fb4ac1a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:44.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4abfff700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ac06c1a0 0x7fb4ac1a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:44.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4abfff700 1 -- 172.21.15.122:0/3094679116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ac1a5150 con 0x7fb4ac105ee0 2024-05-03T17:51:44.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4abfff700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4ac105ee0 0x7fb4ac1a0680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb498000c00 tx=0x7fb4980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:44.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4a9ffb700 1 -- 172.21.15.122:0/3094679116 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498027070 con 0x7fb4ac105ee0 2024-05-03T17:51:44.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3094679116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ac1a53e0 con 0x7fb4ac105ee0 2024-05-03T17:51:44.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4a9ffb700 1 -- 172.21.15.122:0/3094679116 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb498003fe0 con 0x7fb4ac105ee0 2024-05-03T17:51:44.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4a9ffb700 1 -- 172.21.15.122:0/3094679116 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498022e30 con 0x7fb4ac105ee0 2024-05-03T17:51:44.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.277+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3094679116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ac1accc0 con 0x7fb4ac105ee0 2024-05-03T17:51:44.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.278+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3094679116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4ac06a190 con 0x7fb4ac105ee0 2024-05-03T17:51:44.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.280+0000 7fb4a9ffb700 1 -- 172.21.15.122:0/3094679116 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fb49802a070 con 0x7fb4ac105ee0 2024-05-03T17:51:44.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.280+0000 7fb4a9ffb700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb49c0b2880 0x7fb49c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:44.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.280+0000 7fb4b0b3a700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb49c0b2880 0x7fb49c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:44.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.280+0000 7fb4a9ffb700 1 -- 172.21.15.122:0/3094679116 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fb4980e2ab0 con 0x7fb4ac105ee0 2024-05-03T17:51:44.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.281+0000 7fb4b0b3a700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb49c0b2880 0x7fb49c0b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0005950 tx=0x7fb4a000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:44.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.285+0000 7fb4a9ffb700 1 -- 172.21.15.122:0/3094679116 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb498013070 con 0x7fb4ac105ee0 2024-05-03T17:51:44.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:44 smithi122 ceph-mon[29744]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:44.503 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:44 smithi192 ceph-mon[33753]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:44.503 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 17:51:44 smithi192 podman[63560]: 2024-05-03 17:51:44.141054909 +0000 UTC m=+0.018496889 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:51:44.503 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 17:51:44 smithi192 podman[63560]: 2024-05-03 17:51:44.264117425 +0000 UTC m=+0.141559434 container create 67a1d07698113cef427b6253cd7bd8635a13a34286ea0fad7f0b25d719f1876c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T17:51:44.503 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 17:51:44 smithi192 podman[63560]: 2024-05-03 17:51:44.508901916 +0000 UTC m=+0.386343882 container init 67a1d07698113cef427b6253cd7bd8635a13a34286ea0fad7f0b25d719f1876c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-03T17:51:44.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.540+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3094679116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4ac1ace90 con 0x7fb4ac105ee0 2024-05-03T17:51:44.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.540+0000 7fb4a9ffb700 1 -- 172.21.15.122:0/3094679116 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fb4980aad00 con 0x7fb4ac105ee0 2024-05-03T17:51:44.535 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:51:44.535 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-03T17:51:44.536 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:44 smithi149 ceph-mon[33409]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.543+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3094679116 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb49c0b2880 msgr2=0x7fb49c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.543+0000 7fb4b2d9e700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb49c0b2880 0x7fb49c0b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0005950 tx=0x7fb4a000a300 comp rx=0 tx=0).stop 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.543+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3094679116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4ac105ee0 msgr2=0x7fb4ac1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.543+0000 7fb4b2d9e700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4ac105ee0 0x7fb4ac1a0680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb498000c00 tx=0x7fb4980047b0 comp rx=0 tx=0).stop 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.543+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3094679116 shutdown_connections 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.543+0000 7fb4b2d9e700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb49c0b2880 0x7fb49c0b4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.543+0000 7fb4b2d9e700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ac1083d0 0x7fb4ac1a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.543+0000 7fb4b2d9e700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ac06c1a0 0x7fb4ac1a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.543+0000 7fb4b2d9e700 1 --2- 172.21.15.122:0/3094679116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4ac105ee0 0x7fb4ac1a0680 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.543+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3094679116 >> 172.21.15.122:0/3094679116 conn(0x7fb4ac079350 msgr2=0x7fb4ac104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.544+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3094679116 shutdown_connections 2024-05-03T17:51:44.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:44.544+0000 7fb4b2d9e700 1 -- 172.21.15.122:0/3094679116 wait complete. 2024-05-03T17:51:44.540 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 1 2024-05-03T17:51:44.865 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 17:51:44 smithi192 podman[63560]: 2024-05-03 17:51:44.511454411 +0000 UTC m=+0.388896387 container start 67a1d07698113cef427b6253cd7bd8635a13a34286ea0fad7f0b25d719f1876c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-03T17:51:44.865 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 17:51:44 smithi192 ceph-mds[63578]: -- 172.21.15.192:0/155135096 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e777f01a40 con 0x55e777ed5000 2024-05-03T17:51:44.866 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 17:51:44 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l[63574]: starting mds.l at 2024-05-03T17:51:44.866 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 17:51:44 smithi192 bash[63560]: 67a1d07698113cef427b6253cd7bd8635a13a34286ea0fad7f0b25d719f1876c 2024-05-03T17:51:44.866 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 17:51:44 smithi192 systemd[1]: Started Ceph mds.l for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:51:45.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:45 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3094679116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:51:45.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:45 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.192:6832/899389349,v1:172.21.15.192:6833/899389349] up:boot 2024-05-03T17:51:45.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:45 smithi122 ceph-mon[29744]: fsmap 1 up:standby 2024-05-03T17:51:45.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T17:51:45.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:45.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:45.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:45.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:45.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:45.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:45 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:45.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:45 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3094679116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:51:45.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:45 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.192:6832/899389349,v1:172.21.15.192:6833/899389349] up:boot 2024-05-03T17:51:45.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:45 smithi149 ceph-mon[33409]: fsmap 1 up:standby 2024-05-03T17:51:45.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T17:51:45.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:45.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:45.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:45.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:45.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:45.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:45 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:45.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:45 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3094679116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:51:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:45 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.192:6832/899389349,v1:172.21.15.192:6833/899389349] up:boot 2024-05-03T17:51:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:45 smithi192 ceph-mon[33753]: fsmap 1 up:standby 2024-05-03T17:51:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T17:51:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:45.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:45 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:46.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:46 smithi122 ceph-mon[29744]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:46.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:46 smithi122 ceph-mon[29744]: Deploying daemon mds.k on smithi149 2024-05-03T17:51:46.505 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:46 smithi149 ceph-mon[33409]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:46.505 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:46 smithi149 ceph-mon[33409]: Deploying daemon mds.k on smithi149 2024-05-03T17:51:46.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:46 smithi192 ceph-mon[33753]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:46.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:46 smithi192 ceph-mon[33753]: Deploying daemon mds.k on smithi149 2024-05-03T17:51:47.203 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 17:51:46 smithi149 systemd[1]: Starting Ceph mds.k for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:51:47.276 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:51:47.493 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 17:51:47 smithi149 podman[62679]: 2024-05-03 17:51:47.111230507 +0000 UTC m=+0.018366296 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:51:47.550 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 17:51:47 smithi149 podman[62679]: 2024-05-03 17:51:47.221710238 +0000 UTC m=+0.128846016 container create 4a1560d17b8ab054eedd19cd47b3ea148c25cd1a0fe7d54aa37699c2d37dfeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-k, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-03T17:51:47.550 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 17:51:47 smithi149 podman[62679]: 2024-05-03 17:51:47.49929656 +0000 UTC m=+0.406432334 container init 4a1560d17b8ab054eedd19cd47b3ea148c25cd1a0fe7d54aa37699c2d37dfeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T17:51:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:51:47.787 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 17:51:47 smithi149 podman[62679]: 2024-05-03 17:51:47.502280508 +0000 UTC m=+0.409416291 container start 4a1560d17b8ab054eedd19cd47b3ea148c25cd1a0fe7d54aa37699c2d37dfeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T17:51:47.787 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 17:51:47 smithi149 ceph-mds[62698]: -- 172.21.15.149:0/2473260417 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555720647a40 con 0x55572061ac00 2024-05-03T17:51:47.787 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 17:51:47 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-k[62694]: starting mds.k at 2024-05-03T17:51:47.787 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 17:51:47 smithi149 bash[62679]: 4a1560d17b8ab054eedd19cd47b3ea148c25cd1a0fe7d54aa37699c2d37dfeec 2024-05-03T17:51:47.787 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 17:51:47 smithi149 systemd[1]: Started Ceph mds.k for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:51:48.340 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:48 smithi122 ceph-mon[29744]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:48.340 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:48 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:48.340 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:48 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:48.340 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:48 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:48.340 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:48 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:48.340 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:48 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:48.340 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:48 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:48.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:48 smithi149 ceph-mon[33409]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:48.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:48 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:48.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:48 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:48.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:48 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:48.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:48 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:48.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:48 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:48.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:48 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:48.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:48 smithi192 ceph-mon[33753]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:48 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:48 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:48 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:48 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:48 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:48 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:49.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:49 smithi122 ceph-mon[29744]: Deploying daemon mds.j on smithi122 2024-05-03T17:51:49.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:49 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.149:6832/3370166837,v1:172.21.15.149:6833/3370166837] up:boot 2024-05-03T17:51:49.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:49 smithi122 ceph-mon[29744]: fsmap 2 up:standby 2024-05-03T17:51:49.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T17:51:49.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:49 smithi149 ceph-mon[33409]: Deploying daemon mds.j on smithi122 2024-05-03T17:51:49.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:49 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.149:6832/3370166837,v1:172.21.15.149:6833/3370166837] up:boot 2024-05-03T17:51:49.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:49 smithi149 ceph-mon[33409]: fsmap 2 up:standby 2024-05-03T17:51:49.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T17:51:49.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:49 smithi192 ceph-mon[33753]: Deploying daemon mds.j on smithi122 2024-05-03T17:51:49.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:49 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.149:6832/3370166837,v1:172.21.15.149:6833/3370166837] up:boot 2024-05-03T17:51:49.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:49 smithi192 ceph-mon[33753]: fsmap 2 up:standby 2024-05-03T17:51:49.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T17:51:50.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.382+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3215641110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf01023a0 msgr2=0x7f1bf0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:50.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.382+0000 7f1bf4a1f700 1 --2- 172.21.15.122:0/3215641110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf01023a0 0x7f1bf0102820 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009990 tx=0x7f1be4009ca0 comp rx=0 tx=0).stop 2024-05-03T17:51:50.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.382+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3215641110 shutdown_connections 2024-05-03T17:51:50.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.382+0000 7f1bf4a1f700 1 --2- 172.21.15.122:0/3215641110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bf0102d60 0x7f1bf010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:50.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.382+0000 7f1bf4a1f700 1 --2- 172.21.15.122:0/3215641110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf0108410 0x7f1bf01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:50.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.382+0000 7f1bf4a1f700 1 --2- 172.21.15.122:0/3215641110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf01023a0 0x7f1bf0102820 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:50.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.382+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3215641110 >> 172.21.15.122:0/3215641110 conn(0x7f1bf0079350 msgr2=0x7f1bf0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:50.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.382+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3215641110 shutdown_connections 2024-05-03T17:51:50.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.382+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3215641110 wait complete. 2024-05-03T17:51:50.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bf4a1f700 1 Processor -- start 2024-05-03T17:51:50.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bf4a1f700 1 -- start start 2024-05-03T17:51:50.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bf4a1f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bf01023a0 0x7f1bf019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:50.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bf4a1f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf0102d60 0x7f1bf019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:50.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bf4a1f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf0108410 0x7f1bf0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bf4a1f700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf0112550 con 0x7f1bf0102d60 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bf4a1f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf01123d0 con 0x7f1bf01023a0 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bf4a1f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf01126d0 con 0x7f1bf0108410 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bedd9b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf0102d60 0x7f1bf019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bedd9b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf0102d60 0x7f1bf019fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52800/0 (socket says 172.21.15.122:52800) 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bedd9b700 1 -- 172.21.15.122:0/3545633915 learned_addr learned my addr 172.21.15.122:0/3545633915 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1bee59c700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bf01023a0 0x7f1bf019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.383+0000 7f1beed9d700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf0108410 0x7f1bf0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1bedd9b700 1 -- 172.21.15.122:0/3545633915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf0108410 msgr2=0x7f1bf0199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1bedd9b700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf0108410 0x7f1bf0199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1bedd9b700 1 -- 172.21.15.122:0/3545633915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bf01023a0 msgr2=0x7f1bf019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1bedd9b700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bf01023a0 0x7f1bf019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1bedd9b700 1 -- 172.21.15.122:0/3545633915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf019a0e0 con 0x7f1bf0102d60 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1beed9d700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf0108410 0x7f1bf0199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:51:50.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1bedd9b700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf0102d60 0x7f1bf019fd00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1be4000c00 tx=0x7f1be4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:50.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1bdf7fe700 1 -- 172.21.15.122:0/3545633915 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be4027070 con 0x7f1bf0102d60 2024-05-03T17:51:50.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1bdf7fe700 1 -- 172.21.15.122:0/3545633915 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1be4022470 con 0x7f1bf0102d60 2024-05-03T17:51:50.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3545633915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf019a370 con 0x7f1bf0102d60 2024-05-03T17:51:50.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1bdf7fe700 1 -- 172.21.15.122:0/3545633915 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be4017650 con 0x7f1bf0102d60 2024-05-03T17:51:50.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.384+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3545633915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf01a6690 con 0x7f1bf0102d60 2024-05-03T17:51:50.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.385+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3545633915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf006a190 con 0x7f1bf0102d60 2024-05-03T17:51:50.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.387+0000 7f1bdf7fe700 1 -- 172.21.15.122:0/3545633915 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f1be402a070 con 0x7f1bf0102d60 2024-05-03T17:51:50.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.388+0000 7f1bdf7fe700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bd80b2880 0x7f1bd80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:50.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.388+0000 7f1bee59c700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bd80b2880 0x7f1bd80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:50.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.388+0000 7f1bdf7fe700 1 -- 172.21.15.122:0/3545633915 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f1be40e3050 con 0x7f1bf0102d60 2024-05-03T17:51:50.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.389+0000 7f1bee59c700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bd80b2880 0x7f1bd80b4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1bf019aac0 tx=0x7f1be0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:50.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.391+0000 7f1bdf7fe700 1 -- 172.21.15.122:0/3545633915 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1be4013070 con 0x7f1bf0102d60 2024-05-03T17:51:50.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:50 smithi122 ceph-mon[29744]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:50.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:50 smithi149 ceph-mon[33409]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:50.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:50 smithi192 ceph-mon[33753]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:50.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.664+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3545633915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bf01a6860 con 0x7f1bf0102d60 2024-05-03T17:51:50.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.665+0000 7f1bdf7fe700 1 -- 172.21.15.122:0/3545633915 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f1be40ab2a0 con 0x7f1bf0102d60 2024-05-03T17:51:50.660 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:51:50.660 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T17:51:50.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.667+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3545633915 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bd80b2880 msgr2=0x7f1bd80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:50.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.667+0000 7f1bf4a1f700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bd80b2880 0x7f1bd80b4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1bf019aac0 tx=0x7f1be0009260 comp rx=0 tx=0).stop 2024-05-03T17:51:50.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.667+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3545633915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf0102d60 msgr2=0x7f1bf019fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:50.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.667+0000 7f1bf4a1f700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf0102d60 0x7f1bf019fd00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1be4000c00 tx=0x7f1be4009580 comp rx=0 tx=0).stop 2024-05-03T17:51:50.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.667+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3545633915 shutdown_connections 2024-05-03T17:51:50.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.667+0000 7f1bf4a1f700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bd80b2880 0x7f1bd80b4d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:50.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.667+0000 7f1bf4a1f700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf0108410 0x7f1bf0199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:50.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.667+0000 7f1bf4a1f700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bf01023a0 0x7f1bf019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:50.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.667+0000 7f1bf4a1f700 1 --2- 172.21.15.122:0/3545633915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf0102d60 0x7f1bf019fd00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:50.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.667+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3545633915 >> 172.21.15.122:0/3545633915 conn(0x7f1bf0079350 msgr2=0x7f1bf01016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:50.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.668+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3545633915 shutdown_connections 2024-05-03T17:51:50.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:50.668+0000 7f1bf4a1f700 1 -- 172.21.15.122:0/3545633915 wait complete. 2024-05-03T17:51:50.664 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 3 2024-05-03T17:51:51.356 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:51 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3545633915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:51:51.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:51 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3545633915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:51:51.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:51 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3545633915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:51:51.885 INFO:journalctl@ceph.mds.j.smithi122.stdout:May 03 17:51:51 smithi122 podman[67491]: 2024-05-03 17:51:51.867671737 +0000 UTC m=+0.343282964 container start 91076c7dc0123f2c243940a92f6a0aa8cbe78238f20814b1eb399d3d395a3c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-03T17:51:52.210 INFO:journalctl@ceph.mds.j.smithi122.stdout:May 03 17:51:51 smithi122 ceph-mds[67510]: -- 172.21.15.122:0/719388854 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aef799ba40 con 0x55aef796ec00 2024-05-03T17:51:52.210 INFO:journalctl@ceph.mds.j.smithi122.stdout:May 03 17:51:51 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-j[67506]: starting mds.j at 2024-05-03T17:51:52.210 INFO:journalctl@ceph.mds.j.smithi122.stdout:May 03 17:51:51 smithi122 bash[67491]: 91076c7dc0123f2c243940a92f6a0aa8cbe78238f20814b1eb399d3d395a3c05 2024-05-03T17:51:52.210 INFO:journalctl@ceph.mds.j.smithi122.stdout:May 03 17:51:51 smithi122 systemd[1]: Started Ceph mds.j for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:51:52.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:52 smithi149 ceph-mon[33409]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:52.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:52.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:52.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:52.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:52.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:52.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:52.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:52 smithi192 ceph-mon[33753]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:52.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:52 smithi122 ceph-mon[29744]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:52.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:52.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:52.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:52.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:52.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:52.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:53.385 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:51:53.526 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:53 smithi122 ceph-mon[29744]: Deploying daemon mds.d on smithi122 2024-05-03T17:51:53.526 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:53 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.122:6834/2168395467,v1:172.21.15.122:6835/2168395467] up:boot 2024-05-03T17:51:53.526 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:53 smithi122 ceph-mon[29744]: fsmap 3 up:standby 2024-05-03T17:51:53.526 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:53 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T17:51:53.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:53 smithi149 ceph-mon[33409]: Deploying daemon mds.d on smithi122 2024-05-03T17:51:53.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:53 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.122:6834/2168395467,v1:172.21.15.122:6835/2168395467] up:boot 2024-05-03T17:51:53.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:53 smithi149 ceph-mon[33409]: fsmap 3 up:standby 2024-05-03T17:51:53.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:53 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T17:51:53.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:53 smithi192 ceph-mon[33753]: Deploying daemon mds.d on smithi122 2024-05-03T17:51:53.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:53 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.122:6834/2168395467,v1:172.21.15.122:6835/2168395467] up:boot 2024-05-03T17:51:53.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:53 smithi192 ceph-mon[33753]: fsmap 3 up:standby 2024-05-03T17:51:53.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:53 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T17:51:53.708 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:51:54.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:54 smithi122 ceph-mon[29744]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:54.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:54 smithi149 ceph-mon[33409]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:54.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:54 smithi192 ceph-mon[33753]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:55.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.476+0000 7fa4d267f700 1 -- 172.21.15.122:0/3267468567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4cc1037b0 msgr2=0x7fa4cc103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:55.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.476+0000 7fa4d267f700 1 --2- 172.21.15.122:0/3267468567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4cc1037b0 0x7fa4cc103b90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0099c0 tx=0x7fa4bc009cd0 comp rx=0 tx=0).stop 2024-05-03T17:51:55.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.476+0000 7fa4d267f700 1 -- 172.21.15.122:0/3267468567 shutdown_connections 2024-05-03T17:51:55.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.476+0000 7fa4d267f700 1 --2- 172.21.15.122:0/3267468567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4cc108880 0x7fa4cc110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:55.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.476+0000 7fa4d267f700 1 --2- 172.21.15.122:0/3267468567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4cc1037b0 0x7fa4cc103b90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.476+0000 7fa4d267f700 1 --2- 172.21.15.122:0/3267468567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc104160 0x7fa4cc108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.476+0000 7fa4d267f700 1 -- 172.21.15.122:0/3267468567 >> 172.21.15.122:0/3267468567 conn(0x7fa4cc07a580 msgr2=0x7fa4cc07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.476+0000 7fa4d267f700 1 -- 172.21.15.122:0/3267468567 shutdown_connections 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.476+0000 7fa4d267f700 1 -- 172.21.15.122:0/3267468567 wait complete. 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.476+0000 7fa4d267f700 1 Processor -- start 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4d267f700 1 -- start start 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4d267f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4cc1037b0 0x7fa4cc19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4d267f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4cc104160 0x7fa4cc19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4d267f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc108880 0x7fa4cc1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4d267f700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc113740 con 0x7fa4cc104160 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4d267f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc1135c0 con 0x7fa4cc108880 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4d267f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc1138c0 con 0x7fa4cc1037b0 2024-05-03T17:51:55.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4cb7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4cc104160 0x7fa4cc19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4cb7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4cc104160 0x7fa4cc19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52838/0 (socket says 172.21.15.122:52838) 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4cb7fe700 1 -- 172.21.15.122:0/3906203760 learned_addr learned my addr 172.21.15.122:0/3906203760 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4cbfff700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4cc1037b0 0x7fa4cc19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4d0c1c700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc108880 0x7fa4cc1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4cb7fe700 1 -- 172.21.15.122:0/3906203760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4cc1037b0 msgr2=0x7fa4cc19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4cb7fe700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4cc1037b0 0x7fa4cc19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4cb7fe700 1 -- 172.21.15.122:0/3906203760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc108880 msgr2=0x7fa4cc1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4cb7fe700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc108880 0x7fa4cc1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.477+0000 7fa4cb7fe700 1 -- 172.21.15.122:0/3906203760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc19a140 con 0x7fa4cc104160 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.478+0000 7fa4cb7fe700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4cc104160 0x7fa4cc19fd60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000b530 tx=0x7fa4c000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.478+0000 7fa4c97fa700 1 -- 172.21.15.122:0/3906203760 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0015020 con 0x7fa4cc104160 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.478+0000 7fa4d267f700 1 -- 172.21.15.122:0/3906203760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc19a430 con 0x7fa4cc104160 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.478+0000 7fa4c97fa700 1 -- 172.21.15.122:0/3906203760 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4c0011ce0 con 0x7fa4cc104160 2024-05-03T17:51:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.478+0000 7fa4c97fa700 1 -- 172.21.15.122:0/3906203760 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0010e40 con 0x7fa4cc104160 2024-05-03T17:51:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.478+0000 7fa4d267f700 1 -- 172.21.15.122:0/3906203760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc1a6770 con 0x7fa4cc104160 2024-05-03T17:51:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.479+0000 7fa4d267f700 1 -- 172.21.15.122:0/3906203760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4cc10bac0 con 0x7fa4cc104160 2024-05-03T17:51:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.480+0000 7fa4c97fa700 1 -- 172.21.15.122:0/3906203760 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fa4c0018020 con 0x7fa4cc104160 2024-05-03T17:51:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.481+0000 7fa4c97fa700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa4b40b2880 0x7fa4b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:51:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.481+0000 7fa4cbfff700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa4b40b2880 0x7fa4b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:51:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.481+0000 7fa4c97fa700 1 -- 172.21.15.122:0/3906203760 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fa4c00e1b90 con 0x7fa4cc104160 2024-05-03T17:51:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.481+0000 7fa4cbfff700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa4b40b2880 0x7fa4b40b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc19ab20 tx=0x7fa4bc00b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:51:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.484+0000 7fa4c97fa700 1 -- 172.21.15.122:0/3906203760 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa4c001f020 con 0x7fa4cc104160 2024-05-03T17:51:55.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.734+0000 7fa4d267f700 1 -- 172.21.15.122:0/3906203760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4cc06a190 con 0x7fa4cc104160 2024-05-03T17:51:55.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.735+0000 7fa4c97fa700 1 -- 172.21.15.122:0/3906203760 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fa4c00a9d50 con 0x7fa4cc104160 2024-05-03T17:51:55.730 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:51:55.730 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T17:51:55.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 -- 172.21.15.122:0/3906203760 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa4b40b2880 msgr2=0x7fa4b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:55.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa4b40b2880 0x7fa4b40b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc19ab20 tx=0x7fa4bc00b870 comp rx=0 tx=0).stop 2024-05-03T17:51:55.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 -- 172.21.15.122:0/3906203760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4cc104160 msgr2=0x7fa4cc19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:51:55.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4cc104160 0x7fa4cc19fd60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000b530 tx=0x7fa4c000b840 comp rx=0 tx=0).stop 2024-05-03T17:51:55.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 -- 172.21.15.122:0/3906203760 shutdown_connections 2024-05-03T17:51:55.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa4b40b2880 0x7fa4b40b4d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:55.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc108880 0x7fa4cc1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:55.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4cc1037b0 0x7fa4cc19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:55.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 --2- 172.21.15.122:0/3906203760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4cc104160 0x7fa4cc19fd60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:51:55.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 -- 172.21.15.122:0/3906203760 >> 172.21.15.122:0/3906203760 conn(0x7fa4cc07a580 msgr2=0x7fa4cc1099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:51:55.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 -- 172.21.15.122:0/3906203760 shutdown_connections 2024-05-03T17:51:55.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:51:55.737+0000 7fa4d267f700 1 -- 172.21.15.122:0/3906203760 wait complete. 2024-05-03T17:51:55.734 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 4 2024-05-03T17:51:56.335 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:56 smithi122 ceph-mon[29744]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:56.335 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:56 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3906203760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:51:56.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:56 smithi149 ceph-mon[33409]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:56.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:56 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3906203760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:51:56.595 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 17:51:56 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-d[67958]: starting mds.d at 2024-05-03T17:51:56.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:56 smithi192 ceph-mon[33753]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:56.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:56 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3906203760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:51:56.959 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 17:51:56 smithi122 bash[67918]: 53dcd52b8677bbe0d97a2b7996a4019fffbe6e64ae4fcf939877296836195fc4 2024-05-03T17:51:56.960 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 17:51:56 smithi122 systemd[1]: Started Ceph mds.d for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:51:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:57 smithi122 ceph-mon[29744]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:57 smithi122 ceph-mon[29744]: Deploying daemon mds.e on smithi149 2024-05-03T17:51:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:57 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.122:6836/3093239187,v1:172.21.15.122:6837/3093239187] up:boot 2024-05-03T17:51:57.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:57 smithi122 ceph-mon[29744]: fsmap 4 up:standby 2024-05-03T17:51:57.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T17:51:58.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:57 smithi149 ceph-mon[33409]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:58.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:58.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:58.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:58.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:58.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:58.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:58.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:57 smithi149 ceph-mon[33409]: Deploying daemon mds.e on smithi149 2024-05-03T17:51:58.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:57 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.122:6836/3093239187,v1:172.21.15.122:6837/3093239187] up:boot 2024-05-03T17:51:58.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:57 smithi149 ceph-mon[33409]: fsmap 4 up:standby 2024-05-03T17:51:58.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T17:51:58.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:57 smithi192 ceph-mon[33753]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:58.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:58.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:58.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:58.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:58.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:58.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:58.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:57 smithi192 ceph-mon[33753]: Deploying daemon mds.e on smithi149 2024-05-03T17:51:58.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:57 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.122:6836/3093239187,v1:172.21.15.122:6837/3093239187] up:boot 2024-05-03T17:51:58.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:57 smithi192 ceph-mon[33753]: fsmap 4 up:standby 2024-05-03T17:51:58.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T17:51:58.363 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:51:58.618 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:51:59.037 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 17:51:58 smithi149 systemd[1]: Starting Ceph mds.e for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:51:59.416 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 17:51:59 smithi149 podman[62986]: 2024-05-03 17:51:59.15937976 +0000 UTC m=+0.102108123 container create 69442177f39d2420fb56b5ea2bed5d4a16c4ce8478673a0ce5fe0a8506cbd14d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T17:51:59.416 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 17:51:59 smithi149 podman[62986]: 2024-05-03 17:51:59.075563223 +0000 UTC m=+0.018291611 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:51:59.417 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 17:51:59 smithi149 podman[62986]: 2024-05-03 17:51:59.395726653 +0000 UTC m=+0.338455017 container init 69442177f39d2420fb56b5ea2bed5d4a16c4ce8478673a0ce5fe0a8506cbd14d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-e, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-03T17:51:59.417 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 17:51:59 smithi149 podman[62986]: 2024-05-03 17:51:59.398322246 +0000 UTC m=+0.341050610 container start 69442177f39d2420fb56b5ea2bed5d4a16c4ce8478673a0ce5fe0a8506cbd14d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-03T17:51:59.417 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 17:51:59 smithi149 ceph-mds[63005]: -- 172.21.15.149:0/799991872 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b31545a40 con 0x557b31518c00 2024-05-03T17:51:59.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:59 smithi149 ceph-mon[33409]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:59.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:59.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:59.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:59.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:59.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:59.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:51:59 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:51:59.788 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 17:51:59 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-e[63001]: starting mds.e at 2024-05-03T17:51:59.788 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 17:51:59 smithi149 bash[62986]: 69442177f39d2420fb56b5ea2bed5d4a16c4ce8478673a0ce5fe0a8506cbd14d 2024-05-03T17:51:59.788 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 17:51:59 smithi149 systemd[1]: Started Ceph mds.e for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:51:59.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:59 smithi122 ceph-mon[29744]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:51:59.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:59.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:59.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:51:59.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:51:59.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:51:59.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:51:59 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:00.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:59 smithi192 ceph-mon[33753]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:00.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:00.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:00.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:00.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:00.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:00.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:51:59 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:00.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.197+0000 7f07a832b700 1 -- 172.21.15.122:0/3770020033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a0102d50 msgr2=0x7f07a010f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:00.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.197+0000 7f07a832b700 1 --2- 172.21.15.122:0/3770020033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a0102d50 0x7f07a010f240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f079800b0a0 tx=0x7f079800b3b0 comp rx=0 tx=0).stop 2024-05-03T17:52:00.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.197+0000 7f07a832b700 1 -- 172.21.15.122:0/3770020033 shutdown_connections 2024-05-03T17:52:00.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.197+0000 7f07a832b700 1 --2- 172.21.15.122:0/3770020033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a0102d50 0x7f07a010f240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:00.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.197+0000 7f07a832b700 1 --2- 172.21.15.122:0/3770020033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07a0108400 0x7f07a01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:00.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.197+0000 7f07a832b700 1 --2- 172.21.15.122:0/3770020033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a0102390 0x7f07a0102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:00.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.197+0000 7f07a832b700 1 -- 172.21.15.122:0/3770020033 >> 172.21.15.122:0/3770020033 conn(0x7f07a0079310 msgr2=0x7f07a0079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.197+0000 7f07a832b700 1 -- 172.21.15.122:0/3770020033 shutdown_connections 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.197+0000 7f07a832b700 1 -- 172.21.15.122:0/3770020033 wait complete. 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a832b700 1 Processor -- start 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a832b700 1 -- start start 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a832b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a0102390 0x7f07a019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a832b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07a0102d50 0x7f07a019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a832b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a0108400 0x7f07a01a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a832b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07a0112520 con 0x7f07a0102390 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a832b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07a01123a0 con 0x7f07a0108400 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a832b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f07a01126a0 con 0x7f07a0102d50 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a60c7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a0102390 0x7f07a019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a60c7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a0102390 0x7f07a019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46898/0 (socket says 172.21.15.122:46898) 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a60c7700 1 -- 172.21.15.122:0/3017292396 learned_addr learned my addr 172.21.15.122:0/3017292396 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a68c8700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a0108400 0x7f07a01a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a58c6700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07a0102d50 0x7f07a019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a60c7700 1 -- 172.21.15.122:0/3017292396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07a0102d50 msgr2=0x7f07a019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a60c7700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07a0102d50 0x7f07a019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a60c7700 1 -- 172.21.15.122:0/3017292396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a0108400 msgr2=0x7f07a01a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a60c7700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a0108400 0x7f07a01a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.198+0000 7f07a60c7700 1 -- 172.21.15.122:0/3017292396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a01a5e80 con 0x7f07a0102390 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.199+0000 7f07a60c7700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a0102390 0x7f07a019a8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f079000d980 tx=0x7f079000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.199+0000 7f07977fe700 1 -- 172.21.15.122:0/3017292396 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790016070 con 0x7f07a0102390 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.199+0000 7f07a832b700 1 -- 172.21.15.122:0/3017292396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a01a60b0 con 0x7f07a0102390 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.199+0000 7f07977fe700 1 -- 172.21.15.122:0/3017292396 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0790004830 con 0x7f07a0102390 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.199+0000 7f07977fe700 1 -- 172.21.15.122:0/3017292396 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07900054c0 con 0x7f07a0102390 2024-05-03T17:52:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.199+0000 7f07a832b700 1 -- 172.21.15.122:0/3017292396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a01a6610 con 0x7f07a0102390 2024-05-03T17:52:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.200+0000 7f07a832b700 1 -- 172.21.15.122:0/3017292396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a006a190 con 0x7f07a0102390 2024-05-03T17:52:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.201+0000 7f07977fe700 1 -- 172.21.15.122:0/3017292396 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f0790004220 con 0x7f07a0102390 2024-05-03T17:52:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.202+0000 7f07977fe700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f078c0b2880 0x7f078c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.202+0000 7f07a58c6700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f078c0b2880 0x7f078c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:00.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.202+0000 7f07977fe700 1 -- 172.21.15.122:0/3017292396 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f07900e2060 con 0x7f07a0102390 2024-05-03T17:52:00.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.202+0000 7f07a58c6700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f078c0b2880 0x7f078c0b4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f079c005ea0 tx=0x7f079c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:00.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.205+0000 7f07977fe700 1 -- 172.21.15.122:0/3017292396 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f079001f020 con 0x7f07a0102390 2024-05-03T17:52:00.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.455+0000 7f07a832b700 1 -- 172.21.15.122:0/3017292396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07a01a6930 con 0x7f07a0102390 2024-05-03T17:52:00.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.456+0000 7f07977fe700 1 -- 172.21.15.122:0/3017292396 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7f07900aa220 con 0x7f07a0102390 2024-05-03T17:52:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:52:00.451 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T17:52:00.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 -- 172.21.15.122:0/3017292396 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f078c0b2880 msgr2=0x7f078c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f078c0b2880 0x7f078c0b4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f079c005ea0 tx=0x7f079c005e30 comp rx=0 tx=0).stop 2024-05-03T17:52:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 -- 172.21.15.122:0/3017292396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a0102390 msgr2=0x7f07a019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a0102390 0x7f07a019a8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f079000d980 tx=0x7f079000dc90 comp rx=0 tx=0).stop 2024-05-03T17:52:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 -- 172.21.15.122:0/3017292396 shutdown_connections 2024-05-03T17:52:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f078c0b2880 0x7f078c0b4d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a0108400 0x7f07a01a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a0102390 0x7f07a019a8c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 --2- 172.21.15.122:0/3017292396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07a0102d50 0x7f07a019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 -- 172.21.15.122:0/3017292396 >> 172.21.15.122:0/3017292396 conn(0x7f07a0079310 msgr2=0x7f07a0100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 -- 172.21.15.122:0/3017292396 shutdown_connections 2024-05-03T17:52:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:00.458+0000 7f07a832b700 1 -- 172.21.15.122:0/3017292396 wait complete. 2024-05-03T17:52:00.455 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 6 2024-05-03T17:52:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:00 smithi122 ceph-mon[29744]: Deploying daemon mds.f on smithi192 2024-05-03T17:52:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:00 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.149:6834/1470504763,v1:172.21.15.149:6835/1470504763] up:boot 2024-05-03T17:52:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:00 smithi122 ceph-mon[29744]: fsmap 5 up:standby 2024-05-03T17:52:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T17:52:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:00 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3017292396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:00.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:00 smithi192 ceph-mon[33753]: Deploying daemon mds.f on smithi192 2024-05-03T17:52:00.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:00 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.149:6834/1470504763,v1:172.21.15.149:6835/1470504763] up:boot 2024-05-03T17:52:00.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:00 smithi192 ceph-mon[33753]: fsmap 5 up:standby 2024-05-03T17:52:00.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T17:52:00.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:00 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3017292396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:01.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:00 smithi149 ceph-mon[33409]: Deploying daemon mds.f on smithi192 2024-05-03T17:52:01.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:00 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.149:6834/1470504763,v1:172.21.15.149:6835/1470504763] up:boot 2024-05-03T17:52:01.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:00 smithi149 ceph-mon[33409]: fsmap 5 up:standby 2024-05-03T17:52:01.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T17:52:01.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:00 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3017292396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:01.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:01 smithi192 ceph-mon[33753]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:01.866 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 17:52:01 smithi192 systemd[1]: Starting Ceph mds.f for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:52:01.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:01 smithi122 ceph-mon[29744]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:02.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:01 smithi149 ceph-mon[33409]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:02.146 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 17:52:01 smithi192 podman[63867]: 2024-05-03 17:52:01.795669642 +0000 UTC m=+0.017711629 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:52:02.146 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 17:52:01 smithi192 podman[63867]: 2024-05-03 17:52:01.906436883 +0000 UTC m=+0.128478911 container create 81729ac8f34527bdcf18b6a5f4d8dba68c2372c7edc32ccdb703b38d66f1b105 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-03T17:52:02.146 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 17:52:02 smithi192 podman[63867]: 2024-05-03 17:52:02.151182333 +0000 UTC m=+0.373224322 container init 81729ac8f34527bdcf18b6a5f4d8dba68c2372c7edc32ccdb703b38d66f1b105 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True) 2024-05-03T17:52:02.454 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 17:52:02 smithi192 podman[63867]: 2024-05-03 17:52:02.153880508 +0000 UTC m=+0.375922495 container start 81729ac8f34527bdcf18b6a5f4d8dba68c2372c7edc32ccdb703b38d66f1b105 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True) 2024-05-03T17:52:02.454 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 17:52:02 smithi192 ceph-mds[63887]: -- 172.21.15.192:0/2978327502 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5557ac2bfa40 con 0x5557ac292c00 2024-05-03T17:52:02.454 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 17:52:02 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f[63882]: starting mds.f at 2024-05-03T17:52:02.454 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 17:52:02 smithi192 bash[63867]: 81729ac8f34527bdcf18b6a5f4d8dba68c2372c7edc32ccdb703b38d66f1b105 2024-05-03T17:52:02.454 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 17:52:02 smithi192 systemd[1]: Started Ceph mds.f for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:52:03.133 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:52:03.382 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:03.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:03.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:03.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:03.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:03.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:03.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:03.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:03 smithi122 ceph-mon[29744]: Deploying daemon mds.c on smithi192 2024-05-03T17:52:03.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:03 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] up:boot 2024-05-03T17:52:03.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:03 smithi122 ceph-mon[29744]: fsmap 6 up:standby 2024-05-03T17:52:03.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T17:52:03.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:03.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:03.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:03.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:03.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:03.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:03.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:03 smithi192 ceph-mon[33753]: Deploying daemon mds.c on smithi192 2024-05-03T17:52:03.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:03 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] up:boot 2024-05-03T17:52:03.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:03 smithi192 ceph-mon[33753]: fsmap 6 up:standby 2024-05-03T17:52:03.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T17:52:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:03.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:03 smithi149 ceph-mon[33409]: Deploying daemon mds.c on smithi192 2024-05-03T17:52:03.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:03 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] up:boot 2024-05-03T17:52:03.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:03 smithi149 ceph-mon[33409]: fsmap 6 up:standby 2024-05-03T17:52:03.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T17:52:04.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:04 smithi192 ceph-mon[33753]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:04.616 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 17:52:04 smithi192 systemd[1]: Starting Ceph mds.c for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:52:04.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:04 smithi122 ceph-mon[29744]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:04.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:04 smithi149 ceph-mon[33409]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:04.894 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 17:52:04 smithi192 podman[64185]: 2024-05-03 17:52:04.546181363 +0000 UTC m=+0.017965166 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:52:04.894 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 17:52:04 smithi192 podman[64185]: 2024-05-03 17:52:04.663729578 +0000 UTC m=+0.135513408 container create d4bd65c0a1654698980636df1a31bf75d69fed7e37026b630e1541f35cbbf6c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-05-03T17:52:05.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.118+0000 7fb8190f5700 1 -- 172.21.15.122:0/859956658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb814102690 msgr2=0x7fb814102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:05.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.118+0000 7fb8190f5700 1 --2- 172.21.15.122:0/859956658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb814102690 0x7fb814102a70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb8040099c0 tx=0x7fb804009cd0 comp rx=0 tx=0).stop 2024-05-03T17:52:05.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.118+0000 7fb8190f5700 1 -- 172.21.15.122:0/859956658 shutdown_connections 2024-05-03T17:52:05.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.118+0000 7fb8190f5700 1 --2- 172.21.15.122:0/859956658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8141076e0 0x7fb81410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:05.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.118+0000 7fb8190f5700 1 --2- 172.21.15.122:0/859956658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb814102690 0x7fb814102a70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:05.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.118+0000 7fb8190f5700 1 --2- 172.21.15.122:0/859956658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb814103040 0x7fb814107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:05.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.118+0000 7fb8190f5700 1 -- 172.21.15.122:0/859956658 >> 172.21.15.122:0/859956658 conn(0x7fb8140783f0 msgr2=0x7fb814078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:05.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.118+0000 7fb8190f5700 1 -- 172.21.15.122:0/859956658 shutdown_connections 2024-05-03T17:52:05.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.118+0000 7fb8190f5700 1 -- 172.21.15.122:0/859956658 wait complete. 2024-05-03T17:52:05.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb8190f5700 1 Processor -- start 2024-05-03T17:52:05.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb8190f5700 1 -- start start 2024-05-03T17:52:05.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb8190f5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb814102690 0x7fb81419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:05.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb8190f5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb814103040 0x7fb81419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb8190f5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8141076e0 0x7fb814199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb8190f5700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb814112570 con 0x7fb814103040 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb8190f5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb8141123f0 con 0x7fb814102690 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb8190f5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8141126f0 con 0x7fb8141076e0 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb80bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb814103040 0x7fb81419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb80bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb814103040 0x7fb81419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46914/0 (socket says 172.21.15.122:46914) 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb80bfff700 1 -- 172.21.15.122:0/2952871690 learned_addr learned my addr 172.21.15.122:0/2952871690 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb812d9d700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb814102690 0x7fb81419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb81359e700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8141076e0 0x7fb814199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb80bfff700 1 -- 172.21.15.122:0/2952871690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8141076e0 msgr2=0x7fb814199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb80bfff700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8141076e0 0x7fb814199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb80bfff700 1 -- 172.21.15.122:0/2952871690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb814102690 msgr2=0x7fb81419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb80bfff700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb814102690 0x7fb81419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:05.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.119+0000 7fb80bfff700 1 -- 172.21.15.122:0/2952871690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81419a0f0 con 0x7fb814103040 2024-05-03T17:52:05.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.120+0000 7fb80bfff700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb814103040 0x7fb81419fd10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00d950 tx=0x7fb7fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:05.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.120+0000 7fb810d99700 1 -- 172.21.15.122:0/2952871690 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc016070 con 0x7fb814103040 2024-05-03T17:52:05.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.120+0000 7fb8190f5700 1 -- 172.21.15.122:0/2952871690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81419a3e0 con 0x7fb814103040 2024-05-03T17:52:05.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.120+0000 7fb810d99700 1 -- 172.21.15.122:0/2952871690 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7fc004830 con 0x7fb814103040 2024-05-03T17:52:05.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.120+0000 7fb810d99700 1 -- 172.21.15.122:0/2952871690 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc005520 con 0x7fb814103040 2024-05-03T17:52:05.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.120+0000 7fb8190f5700 1 -- 172.21.15.122:0/2952871690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8141a66a0 con 0x7fb814103040 2024-05-03T17:52:05.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.121+0000 7fb8190f5700 1 -- 172.21.15.122:0/2952871690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb81406a190 con 0x7fb814103040 2024-05-03T17:52:05.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.122+0000 7fb810d99700 1 -- 172.21.15.122:0/2952871690 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fb7fc004220 con 0x7fb814103040 2024-05-03T17:52:05.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.123+0000 7fb810d99700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb8000b2880 0x7fb8000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:05.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.123+0000 7fb812d9d700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb8000b2880 0x7fb8000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:05.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.123+0000 7fb810d99700 1 -- 172.21.15.122:0/2952871690 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fb7fc0e1f30 con 0x7fb814103040 2024-05-03T17:52:05.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.123+0000 7fb812d9d700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb8000b2880 0x7fb8000b4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb81419aad0 tx=0x7fb804005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:05.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.126+0000 7fb810d99700 1 -- 172.21.15.122:0/2952871690 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb7fc01f020 con 0x7fb814103040 2024-05-03T17:52:05.365 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 17:52:04 smithi192 podman[64185]: 2024-05-03 17:52:04.900185419 +0000 UTC m=+0.371969204 container init d4bd65c0a1654698980636df1a31bf75d69fed7e37026b630e1541f35cbbf6c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-03T17:52:05.366 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 17:52:04 smithi192 podman[64185]: 2024-05-03 17:52:04.902748264 +0000 UTC m=+0.374532049 container start d4bd65c0a1654698980636df1a31bf75d69fed7e37026b630e1541f35cbbf6c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T17:52:05.366 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 17:52:04 smithi192 ceph-mds[64205]: -- 172.21.15.192:0/1182273859 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5643d0119a40 con 0x5643d00ec800 2024-05-03T17:52:05.366 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 17:52:04 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c[64201]: starting mds.c at 2024-05-03T17:52:05.366 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 17:52:04 smithi192 bash[64185]: d4bd65c0a1654698980636df1a31bf75d69fed7e37026b630e1541f35cbbf6c2 2024-05-03T17:52:05.366 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 17:52:04 smithi192 systemd[1]: Started Ceph mds.c for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:52:05.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.382+0000 7fb8190f5700 1 -- 172.21.15.122:0/2952871690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb81402e510 con 0x7fb814103040 2024-05-03T17:52:05.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.382+0000 7fb810d99700 1 -- 172.21.15.122:0/2952871690 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6361 (secure 0 0 0) 0x7fb7fc0aa0f0 con 0x7fb814103040 2024-05-03T17:52:05.378 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:52:05.379 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-03T17:52:05.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 -- 172.21.15.122:0/2952871690 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb8000b2880 msgr2=0x7fb8000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:05.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb8000b2880 0x7fb8000b4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb81419aad0 tx=0x7fb804005c40 comp rx=0 tx=0).stop 2024-05-03T17:52:05.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 -- 172.21.15.122:0/2952871690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb814103040 msgr2=0x7fb81419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:05.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb814103040 0x7fb81419fd10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00d950 tx=0x7fb7fc00dc60 comp rx=0 tx=0).stop 2024-05-03T17:52:05.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 -- 172.21.15.122:0/2952871690 shutdown_connections 2024-05-03T17:52:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb8000b2880 0x7fb8000b4d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8141076e0 0x7fb814199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb814102690 0x7fb81419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 --2- 172.21.15.122:0/2952871690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb814103040 0x7fb81419fd10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 -- 172.21.15.122:0/2952871690 >> 172.21.15.122:0/2952871690 conn(0x7fb8140783f0 msgr2=0x7fb81406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 -- 172.21.15.122:0/2952871690 shutdown_connections 2024-05-03T17:52:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:05.385+0000 7fb8190f5700 1 -- 172.21.15.122:0/2952871690 wait complete. 2024-05-03T17:52:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-05-03T17:52:06.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:06.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:06.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:06.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:06.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:06.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:06.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:06.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: Deploying daemon mds.h on smithi149 2024-05-03T17:52:06.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] up:boot 2024-05-03T17:52:06.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: fsmap 7 up:standby 2024-05-03T17:52:06.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T17:52:06.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:06 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2952871690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:06.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:06.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:06.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:06.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:06.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:06.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:06.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:06.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: Deploying daemon mds.h on smithi149 2024-05-03T17:52:06.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] up:boot 2024-05-03T17:52:06.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: fsmap 7 up:standby 2024-05-03T17:52:06.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T17:52:06.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:06 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2952871690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:06.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:06.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:06.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:06.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:06.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:06.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:06.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:06.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: Deploying daemon mds.h on smithi149 2024-05-03T17:52:06.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] up:boot 2024-05-03T17:52:06.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: fsmap 7 up:standby 2024-05-03T17:52:06.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T17:52:06.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:06 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2952871690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:07.513 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 17:52:07 smithi149 systemd[1]: Starting Ceph mds.h for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:52:07.778 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 17:52:07 smithi149 podman[63289]: 2024-05-03 17:52:07.42215545 +0000 UTC m=+0.018869949 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:52:07.779 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 17:52:07 smithi149 podman[63289]: 2024-05-03 17:52:07.548041877 +0000 UTC m=+0.144756435 container create d43d175fadb978164a6e616568282519957830f56de96f023c4b102ff2bef9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS) 2024-05-03T17:52:07.779 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 17:52:07 smithi149 podman[63289]: 2024-05-03 17:52:07.784123668 +0000 UTC m=+0.380838167 container init d43d175fadb978164a6e616568282519957830f56de96f023c4b102ff2bef9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-03T17:52:08.034 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 17:52:07 smithi149 podman[63289]: 2024-05-03 17:52:07.786853684 +0000 UTC m=+0.383568183 container start d43d175fadb978164a6e616568282519957830f56de96f023c4b102ff2bef9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T17:52:08.034 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 17:52:07 smithi149 ceph-mds[63308]: -- 172.21.15.149:0/184141871 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55adf6ecda40 con 0x55adf6ea0800 2024-05-03T17:52:08.034 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 17:52:07 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h[63304]: starting mds.h at 2024-05-03T17:52:08.034 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 17:52:07 smithi149 bash[63289]: d43d175fadb978164a6e616568282519957830f56de96f023c4b102ff2bef9ad 2024-05-03T17:52:08.034 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 17:52:07 smithi149 systemd[1]: Started Ceph mds.h for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:52:08.079 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:52:08.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:08 smithi149 ceph-mon[33409]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:08.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:08 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:08.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:08 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:08.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:08 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:08.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:08 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:08.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:08 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:08.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:08 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:08.334 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:08.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:08 smithi192 ceph-mon[33753]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:08.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:08 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:08.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:08 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:08.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:08 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:08.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:08 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:08.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:08 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:08.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:08 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:08.455 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:08 smithi122 ceph-mon[29744]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:08.456 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:08 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:08.456 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:08 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:08.456 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:08 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:08.456 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:08 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:08.456 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:08 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:08.456 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:08 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:09.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:09 smithi192 ceph-mon[33753]: Deploying daemon mds.i on smithi192 2024-05-03T17:52:09.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:09 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.149:6836/2599524712,v1:172.21.15.149:6837/2599524712] up:boot 2024-05-03T17:52:09.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:09 smithi192 ceph-mon[33753]: fsmap 8 up:standby 2024-05-03T17:52:09.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:09 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T17:52:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:09 smithi122 ceph-mon[29744]: Deploying daemon mds.i on smithi192 2024-05-03T17:52:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:09 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.149:6836/2599524712,v1:172.21.15.149:6837/2599524712] up:boot 2024-05-03T17:52:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:09 smithi122 ceph-mon[29744]: fsmap 8 up:standby 2024-05-03T17:52:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:09 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T17:52:09.304 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:09 smithi149 ceph-mon[33409]: Deploying daemon mds.i on smithi192 2024-05-03T17:52:09.305 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:09 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.149:6836/2599524712,v1:172.21.15.149:6837/2599524712] up:boot 2024-05-03T17:52:09.305 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:09 smithi149 ceph-mon[33409]: fsmap 8 up:standby 2024-05-03T17:52:09.305 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:09 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T17:52:10.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.034+0000 7fce11a27700 1 -- 172.21.15.122:0/3913897515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce0c108410 msgr2=0x7fce0c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:10.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.034+0000 7fce11a27700 1 --2- 172.21.15.122:0/3913897515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce0c108410 0x7fce0c1087f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00b0a0 tx=0x7fcdfc00b3b0 comp rx=0 tx=0).stop 2024-05-03T17:52:10.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.034+0000 7fce11a27700 1 -- 172.21.15.122:0/3913897515 shutdown_connections 2024-05-03T17:52:10.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.034+0000 7fce11a27700 1 --2- 172.21.15.122:0/3913897515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce0c102d60 0x7fce0c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:10.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.034+0000 7fce11a27700 1 --2- 172.21.15.122:0/3913897515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce0c108410 0x7fce0c1087f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:10.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.034+0000 7fce11a27700 1 --2- 172.21.15.122:0/3913897515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce0c1023a0 0x7fce0c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:10.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.034+0000 7fce11a27700 1 -- 172.21.15.122:0/3913897515 >> 172.21.15.122:0/3913897515 conn(0x7fce0c079350 msgr2=0x7fce0c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:10.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.034+0000 7fce11a27700 1 -- 172.21.15.122:0/3913897515 shutdown_connections 2024-05-03T17:52:10.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.034+0000 7fce11a27700 1 -- 172.21.15.122:0/3913897515 wait complete. 2024-05-03T17:52:10.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.034+0000 7fce11a27700 1 Processor -- start 2024-05-03T17:52:10.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce11a27700 1 -- start start 2024-05-03T17:52:10.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce11a27700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce0c1023a0 0x7fce0c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:10.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce11a27700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce0c102d60 0x7fce0c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:10.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce11a27700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce0c108410 0x7fce0c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:10.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce11a27700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c112560 con 0x7fce0c102d60 2024-05-03T17:52:10.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce11a27700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1123e0 con 0x7fce0c1023a0 2024-05-03T17:52:10.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce11a27700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1126e0 con 0x7fce0c108410 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0a7fc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce0c102d60 0x7fce0c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0a7fc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce0c102d60 0x7fce0c1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33556/0 (socket says 172.21.15.122:33556) 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0a7fc700 1 -- 172.21.15.122:0/1273670252 learned_addr learned my addr 172.21.15.122:0/1273670252 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0b7fe700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce0c108410 0x7fce0c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0affd700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce0c1023a0 0x7fce0c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0b7fe700 1 -- 172.21.15.122:0/1273670252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce0c1023a0 msgr2=0x7fce0c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0b7fe700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce0c1023a0 0x7fce0c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0b7fe700 1 -- 172.21.15.122:0/1273670252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce0c102d60 msgr2=0x7fce0c1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0b7fe700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce0c102d60 0x7fce0c1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0b7fe700 1 -- 172.21.15.122:0/1273670252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c1a8080 con 0x7fce0c108410 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0a7fc700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce0c102d60 0x7fce0c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0affd700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce0c1023a0 0x7fce0c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T17:52:10.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.035+0000 7fce0b7fe700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce0c108410 0x7fce0c104b00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce0400e970 tx=0x7fce0400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:10.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.036+0000 7fce10a25700 1 -- 172.21.15.122:0/1273670252 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04004220 con 0x7fce0c108410 2024-05-03T17:52:10.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.036+0000 7fce11a27700 1 -- 172.21.15.122:0/1273670252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c1a8370 con 0x7fce0c108410 2024-05-03T17:52:10.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.036+0000 7fce10a25700 1 -- 172.21.15.122:0/1273670252 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce040043c0 con 0x7fce0c108410 2024-05-03T17:52:10.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.036+0000 7fce10a25700 1 -- 172.21.15.122:0/1273670252 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04010700 con 0x7fce0c108410 2024-05-03T17:52:10.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.036+0000 7fce11a27700 1 -- 172.21.15.122:0/1273670252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c1a88b0 con 0x7fce0c108410 2024-05-03T17:52:10.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.037+0000 7fce11a27700 1 -- 172.21.15.122:0/1273670252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0c06a190 con 0x7fce0c108410 2024-05-03T17:52:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.038+0000 7fce10a25700 1 -- 172.21.15.122:0/1273670252 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fce04004750 con 0x7fce0c108410 2024-05-03T17:52:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.039+0000 7fce10a25700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcdf80b2880 0x7fcdf80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.039+0000 7fce0affd700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcdf80b2880 0x7fcdf80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.039+0000 7fce10a25700 1 -- 172.21.15.122:0/1273670252 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fce040e1c60 con 0x7fce0c108410 2024-05-03T17:52:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.039+0000 7fce0affd700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcdf80b2880 0x7fcdf80b4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009fd0 tx=0x7fcdfc00b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:10.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.043+0000 7fce10a25700 1 -- 172.21.15.122:0/1273670252 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fce040a9e10 con 0x7fce0c108410 2024-05-03T17:52:10.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:10 smithi192 ceph-mon[33753]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:10.116 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:10 smithi192 systemd[1]: Starting Ceph mds.i for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:52:10.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:10 smithi122 ceph-mon[29744]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.298+0000 7fce11a27700 1 -- 172.21.15.122:0/1273670252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce0c1a8b70 con 0x7fce0c108410 2024-05-03T17:52:10.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.299+0000 7fce10a25700 1 -- 172.21.15.122:0/1273670252 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7189 (secure 0 0 0) 0x7fce040a9710 con 0x7fce0c108410 2024-05-03T17:52:10.294 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:52:10.294 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24410,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-03T17:52:10.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 -- 172.21.15.122:0/1273670252 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcdf80b2880 msgr2=0x7fcdf80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcdf80b2880 0x7fcdf80b4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009fd0 tx=0x7fcdfc00b700 comp rx=0 tx=0).stop 2024-05-03T17:52:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 -- 172.21.15.122:0/1273670252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce0c108410 msgr2=0x7fce0c104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce0c108410 0x7fce0c104b00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce0400e970 tx=0x7fce0400ec80 comp rx=0 tx=0).stop 2024-05-03T17:52:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 -- 172.21.15.122:0/1273670252 shutdown_connections 2024-05-03T17:52:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcdf80b2880 0x7fcdf80b4d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce0c108410 0x7fce0c104b00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce0c1023a0 0x7fce0c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 --2- 172.21.15.122:0/1273670252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce0c102d60 0x7fce0c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 -- 172.21.15.122:0/1273670252 >> 172.21.15.122:0/1273670252 conn(0x7fce0c079350 msgr2=0x7fce0c1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 -- 172.21.15.122:0/1273670252 shutdown_connections 2024-05-03T17:52:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:10.301+0000 7fce11a27700 1 -- 172.21.15.122:0/1273670252 wait complete. 2024-05-03T17:52:10.298 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 9 2024-05-03T17:52:10.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:10 smithi149 ceph-mon[33409]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:10.559 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:10 smithi192 podman[64496]: 2024-05-03 17:52:10.173474291 +0000 UTC m=+0.018455821 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:52:10.559 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:10 smithi192 podman[64496]: 2024-05-03 17:52:10.317483552 +0000 UTC m=+0.162465100 container create b336f769e1d5f767ab8c3830994a82cc645a6af16d5c093a2ba7bf388b0fc7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-03T17:52:10.559 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:10 smithi192 podman[64496]: 2024-05-03 17:52:10.564980435 +0000 UTC m=+0.409961941 container init b336f769e1d5f767ab8c3830994a82cc645a6af16d5c093a2ba7bf388b0fc7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-03T17:52:10.866 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:10 smithi192 podman[64496]: 2024-05-03 17:52:10.567834463 +0000 UTC m=+0.412815970 container start b336f769e1d5f767ab8c3830994a82cc645a6af16d5c093a2ba7bf388b0fc7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430) 2024-05-03T17:52:10.866 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:10 smithi192 ceph-mds[64515]: -- 172.21.15.192:0/2523934839 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559357f4da40 con 0x559357f20800 2024-05-03T17:52:10.866 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:10 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i[64511]: starting mds.i at 2024-05-03T17:52:10.866 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:10 smithi192 bash[64496]: b336f769e1d5f767ab8c3830994a82cc645a6af16d5c093a2ba7bf388b0fc7ba 2024-05-03T17:52:10.866 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:10 smithi192 systemd[1]: Started Ceph mds.i for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:52:11.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:11 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/1273670252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:11.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:11.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:11.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:11.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:11.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:11.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:11 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:11.366 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:11 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1273670252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:11.366 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:11.366 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:11.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:11.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:11.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:11.367 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:11 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:11.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:11 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/1273670252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:11.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:11.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:11.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:11.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:11.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:11.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:11 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:12.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:12 smithi192 ceph-mon[33753]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:12.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:12 smithi192 ceph-mon[33753]: Deploying daemon mds.g on smithi122 2024-05-03T17:52:12.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:12 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] up:boot 2024-05-03T17:52:12.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:12 smithi192 ceph-mon[33753]: fsmap 9 up:standby 2024-05-03T17:52:12.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T17:52:12.390 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:12 smithi122 ceph-mon[29744]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:12.390 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:12 smithi122 ceph-mon[29744]: Deploying daemon mds.g on smithi122 2024-05-03T17:52:12.390 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:12 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] up:boot 2024-05-03T17:52:12.390 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:12 smithi122 ceph-mon[29744]: fsmap 9 up:standby 2024-05-03T17:52:12.390 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T17:52:12.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:12 smithi149 ceph-mon[33409]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:12.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:12 smithi149 ceph-mon[33409]: Deploying daemon mds.g on smithi122 2024-05-03T17:52:12.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:12 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] up:boot 2024-05-03T17:52:12.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:12 smithi149 ceph-mon[33409]: fsmap 9 up:standby 2024-05-03T17:52:12.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T17:52:12.960 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 17:52:12 smithi122 systemd[1]: Starting Ceph mds.g for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:52:13.022 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:52:13.305 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 17:52:13 smithi122 podman[68751]: 2024-05-03 17:52:12.995112588 +0000 UTC m=+0.018383540 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:52:13.306 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 17:52:13 smithi122 podman[68751]: 2024-05-03 17:52:13.102088729 +0000 UTC m=+0.125359652 container create c5e8610b8cf310ff5d326a4f754c0212eb6730753e20edc0bbbf73e1b2600391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-g, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T17:52:13.583 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 17:52:13 smithi122 podman[68751]: 2024-05-03 17:52:13.488592247 +0000 UTC m=+0.511863174 container init c5e8610b8cf310ff5d326a4f754c0212eb6730753e20edc0bbbf73e1b2600391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-05-03T17:52:13.584 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 17:52:13 smithi122 podman[68751]: 2024-05-03 17:52:13.491117846 +0000 UTC m=+0.514388773 container start c5e8610b8cf310ff5d326a4f754c0212eb6730753e20edc0bbbf73e1b2600391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-g, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T17:52:13.584 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 17:52:13 smithi122 ceph-mds[68819]: -- 172.21.15.122:0/839195216 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c88f5cda40 con 0x55c88f5a0800 2024-05-03T17:52:13.584 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 17:52:13 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-g[68796]: starting mds.g at 2024-05-03T17:52:13.646 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:13.894 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 17:52:13 smithi122 bash[68751]: c5e8610b8cf310ff5d326a4f754c0212eb6730753e20edc0bbbf73e1b2600391 2024-05-03T17:52:13.894 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 17:52:13 smithi122 systemd[1]: Started Ceph mds.g for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:52:14.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:14 smithi122 ceph-mon[29744]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:14.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:14 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:14.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:14 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:14.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:14 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:14.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:14 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:14.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:14 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:14.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:14 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:14.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:14 smithi192 ceph-mon[33753]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:14 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:14 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:14 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:14 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:14 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:14.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:14 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:14.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:14 smithi149 ceph-mon[33409]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:14.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:14 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:14.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:14 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:14.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:14 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:14.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:14 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:14.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:14 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:14.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:14 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:15.358 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:15 smithi122 ceph-mon[29744]: Deploying daemon mds.a on smithi122 2024-05-03T17:52:15.358 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:15 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.122:6838/555236113,v1:172.21.15.122:6839/555236113] up:boot 2024-05-03T17:52:15.358 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:15 smithi122 ceph-mon[29744]: fsmap 10 up:standby 2024-05-03T17:52:15.358 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:15 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T17:52:15.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:15 smithi192 ceph-mon[33753]: Deploying daemon mds.a on smithi122 2024-05-03T17:52:15.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:15 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.122:6838/555236113,v1:172.21.15.122:6839/555236113] up:boot 2024-05-03T17:52:15.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:15 smithi192 ceph-mon[33753]: fsmap 10 up:standby 2024-05-03T17:52:15.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:15 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T17:52:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.463+0000 7ff0878f3700 1 -- 172.21.15.122:0/2559713502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff080104100 msgr2=0x7ff080108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.463+0000 7ff0878f3700 1 --2- 172.21.15.122:0/2559713502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff080104100 0x7ff080108120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff0700099c0 tx=0x7ff070009cd0 comp rx=0 tx=0).stop 2024-05-03T17:52:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.463+0000 7ff0878f3700 1 -- 172.21.15.122:0/2559713502 shutdown_connections 2024-05-03T17:52:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.463+0000 7ff0878f3700 1 --2- 172.21.15.122:0/2559713502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080108850 0x7ff0801103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.463+0000 7ff0878f3700 1 --2- 172.21.15.122:0/2559713502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff080103750 0x7ff080103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.463+0000 7ff0878f3700 1 --2- 172.21.15.122:0/2559713502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff080104100 0x7ff080108120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:15.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.463+0000 7ff0878f3700 1 -- 172.21.15.122:0/2559713502 >> 172.21.15.122:0/2559713502 conn(0x7ff08007a600 msgr2=0x7ff08007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:15.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.463+0000 7ff0878f3700 1 -- 172.21.15.122:0/2559713502 shutdown_connections 2024-05-03T17:52:15.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.463+0000 7ff0878f3700 1 -- 172.21.15.122:0/2559713502 wait complete. 2024-05-03T17:52:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.463+0000 7ff0878f3700 1 Processor -- start 2024-05-03T17:52:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.463+0000 7ff0878f3700 1 -- start start 2024-05-03T17:52:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff0878f3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080103750 0x7ff08019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff0878f3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff080104100 0x7ff08019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff0878f3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff080108850 0x7ff080199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff0878f3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff080113630 con 0x7ff080104100 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff0878f3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801134b0 con 0x7ff080108850 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff0878f3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801137b0 con 0x7ff080103750 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff084e8e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff080104100 0x7ff08019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff084e8e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff080104100 0x7ff08019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33606/0 (socket says 172.21.15.122:33606) 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff084e8e700 1 -- 172.21.15.122:0/2301258510 learned_addr learned my addr 172.21.15.122:0/2301258510 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff08568f700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080103750 0x7ff08019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff085e90700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff080108850 0x7ff080199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff084e8e700 1 -- 172.21.15.122:0/2301258510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080103750 msgr2=0x7ff08019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff084e8e700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080103750 0x7ff08019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff084e8e700 1 -- 172.21.15.122:0/2301258510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff080108850 msgr2=0x7ff080199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff084e8e700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff080108850 0x7ff080199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:15.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff084e8e700 1 -- 172.21.15.122:0/2301258510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08019a070 con 0x7ff080104100 2024-05-03T17:52:15.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff084e8e700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff080104100 0x7ff08019fc90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0700098e0 tx=0x7ff0700057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:15.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff0767fc700 1 -- 172.21.15.122:0/2301258510 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070027070 con 0x7ff080104100 2024-05-03T17:52:15.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.464+0000 7ff0767fc700 1 -- 172.21.15.122:0/2301258510 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff070003f20 con 0x7ff080104100 2024-05-03T17:52:15.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.465+0000 7ff0878f3700 1 -- 172.21.15.122:0/2301258510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08019a300 con 0x7ff080104100 2024-05-03T17:52:15.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.465+0000 7ff0767fc700 1 -- 172.21.15.122:0/2301258510 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070022ca0 con 0x7ff080104100 2024-05-03T17:52:15.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.465+0000 7ff0878f3700 1 -- 172.21.15.122:0/2301258510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0801a6620 con 0x7ff080104100 2024-05-03T17:52:15.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.465+0000 7ff0878f3700 1 -- 172.21.15.122:0/2301258510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff080104be0 con 0x7ff080104100 2024-05-03T17:52:15.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.467+0000 7ff0767fc700 1 -- 172.21.15.122:0/2301258510 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff07002a070 con 0x7ff080104100 2024-05-03T17:52:15.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.467+0000 7ff0767fc700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff06c0b2880 0x7ff06c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:15.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.467+0000 7ff08568f700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff06c0b2880 0x7ff06c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:15.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.468+0000 7ff0767fc700 1 -- 172.21.15.122:0/2301258510 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7ff0700e33e0 con 0x7ff080104100 2024-05-03T17:52:15.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.468+0000 7ff08568f700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff06c0b2880 0x7ff06c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff08019aa50 tx=0x7ff07c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:15.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.471+0000 7ff0767fc700 1 -- 172.21.15.122:0/2301258510 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff070031020 con 0x7ff080104100 2024-05-03T17:52:15.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:15 smithi149 ceph-mon[33409]: Deploying daemon mds.a on smithi122 2024-05-03T17:52:15.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:15 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.122:6838/555236113,v1:172.21.15.122:6839/555236113] up:boot 2024-05-03T17:52:15.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:15 smithi149 ceph-mon[33409]: fsmap 10 up:standby 2024-05-03T17:52:15.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:15 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T17:52:15.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.742+0000 7ff0878f3700 1 -- 172.21.15.122:0/2301258510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff08002e510 con 0x7ff080104100 2024-05-03T17:52:15.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.742+0000 7ff0767fc700 1 -- 172.21.15.122:0/2301258510 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8845 (secure 0 0 0) 0x7ff070013070 con 0x7ff080104100 2024-05-03T17:52:15.738 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:52:15.738 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/555236113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":555236113},{"type":"v1","addr":"172.21.15.122:6839","nonce":555236113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24410,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/1429818440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1429818440},{"type":"v1","addr":"172.21.15.192:6839","nonce":1429818440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-03T17:52:15.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 -- 172.21.15.122:0/2301258510 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff06c0b2880 msgr2=0x7ff06c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:15.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff06c0b2880 0x7ff06c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff08019aa50 tx=0x7ff07c00b410 comp rx=0 tx=0).stop 2024-05-03T17:52:15.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 -- 172.21.15.122:0/2301258510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff080104100 msgr2=0x7ff08019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:15.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff080104100 0x7ff08019fc90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0700098e0 tx=0x7ff0700057c0 comp rx=0 tx=0).stop 2024-05-03T17:52:15.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 -- 172.21.15.122:0/2301258510 shutdown_connections 2024-05-03T17:52:15.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff06c0b2880 0x7ff06c0b4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:15.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff080108850 0x7ff080199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:15.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080103750 0x7ff08019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:15.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 --2- 172.21.15.122:0/2301258510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff080104100 0x7ff08019fc90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:15.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 -- 172.21.15.122:0/2301258510 >> 172.21.15.122:0/2301258510 conn(0x7ff08007a600 msgr2=0x7ff080101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:15.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 -- 172.21.15.122:0/2301258510 shutdown_connections 2024-05-03T17:52:15.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:15.745+0000 7ff0878f3700 1 -- 172.21.15.122:0/2301258510 wait complete. 2024-05-03T17:52:15.741 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 11 2024-05-03T17:52:16.214 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:16 smithi122 ceph-mon[29744]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:16.214 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:16 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2301258510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:16.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:16 smithi192 ceph-mon[33753]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:16.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:16 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2301258510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:16.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:16 smithi149 ceph-mon[33409]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:16.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:16 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2301258510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:17.670 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 17:52:17 smithi122 systemd[1]: Starting Ceph mds.a for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:52:17.941 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 17:52:17 smithi122 podman[69226]: 2024-05-03 17:52:17.578643805 +0000 UTC m=+0.018009595 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:52:17.941 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 17:52:17 smithi122 podman[69226]: 2024-05-03 17:52:17.693616517 +0000 UTC m=+0.132982294 container create 818cdb0c4598034d4b01f9ead224f8beaeee419bd00ba2f185df68a15eea2ce7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-05-03T17:52:17.941 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 17:52:17 smithi122 podman[69226]: 2024-05-03 17:52:17.946639595 +0000 UTC m=+0.386005372 container init 818cdb0c4598034d4b01f9ead224f8beaeee419bd00ba2f185df68a15eea2ce7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430) 2024-05-03T17:52:18.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:18 smithi122 ceph-mon[29744]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:18.210 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 17:52:17 smithi122 podman[69226]: 2024-05-03 17:52:17.949337861 +0000 UTC m=+0.388703640 container start 818cdb0c4598034d4b01f9ead224f8beaeee419bd00ba2f185df68a15eea2ce7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-03T17:52:18.210 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 17:52:17 smithi122 ceph-mds[69245]: -- 172.21.15.122:0/2654636588 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eafa481a40 con 0x55eafa455000 2024-05-03T17:52:18.210 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 17:52:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-a[69241]: starting mds.a at 2024-05-03T17:52:18.210 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 17:52:18 smithi122 bash[69226]: 818cdb0c4598034d4b01f9ead224f8beaeee419bd00ba2f185df68a15eea2ce7 2024-05-03T17:52:18.210 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 17:52:18 smithi122 systemd[1]: Started Ceph mds.a for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:52:18.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:18 smithi192 ceph-mon[33753]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:18.523 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:52:18.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:18 smithi149 ceph-mon[33409]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:18.770 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:19.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:19 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.122:6840/1783292878,v1:172.21.15.122:6841/1783292878] up:boot 2024-05-03T17:52:19.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:19 smithi122 ceph-mon[29744]: fsmap 11 up:standby 2024-05-03T17:52:19.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T17:52:19.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:19.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:19.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:19.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:19.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:19.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:19.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:19 smithi122 ceph-mon[29744]: Deploying daemon mds.b on smithi149 2024-05-03T17:52:19.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:19 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.122:6840/1783292878,v1:172.21.15.122:6841/1783292878] up:boot 2024-05-03T17:52:19.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:19 smithi192 ceph-mon[33753]: fsmap 11 up:standby 2024-05-03T17:52:19.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T17:52:19.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:19.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:19.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:19.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:19.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:19.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:19.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:19 smithi192 ceph-mon[33753]: Deploying daemon mds.b on smithi149 2024-05-03T17:52:19.492 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:19 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.122:6840/1783292878,v1:172.21.15.122:6841/1783292878] up:boot 2024-05-03T17:52:19.492 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:19 smithi149 ceph-mon[33409]: fsmap 11 up:standby 2024-05-03T17:52:19.492 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T17:52:19.492 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:19.492 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:19.492 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:19.492 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T17:52:19.493 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T17:52:19.493 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:19.493 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:19 smithi149 ceph-mon[33409]: Deploying daemon mds.b on smithi149 2024-05-03T17:52:20.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:20 smithi149 ceph-mon[33409]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:20.429 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:20 smithi122 ceph-mon[29744]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:20.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.435+0000 7f6918458700 1 -- 172.21.15.122:0/3531011020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6910102870 msgr2=0x7f691010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:20.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.435+0000 7f6918458700 1 --2- 172.21.15.122:0/3531011020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6910102870 0x7f691010e9a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f690800b0d0 tx=0x7f690800b3e0 comp rx=0 tx=0).stop 2024-05-03T17:52:20.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.435+0000 7f6918458700 1 -- 172.21.15.122:0/3531011020 shutdown_connections 2024-05-03T17:52:20.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.435+0000 7f6918458700 1 --2- 172.21.15.122:0/3531011020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6910102870 0x7f691010e9a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:20.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.435+0000 7f6918458700 1 --2- 172.21.15.122:0/3531011020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6910107b80 0x7f6910107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.435+0000 7f6918458700 1 --2- 172.21.15.122:0/3531011020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6910101eb0 0x7f6910102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.435+0000 7f6918458700 1 -- 172.21.15.122:0/3531011020 >> 172.21.15.122:0/3531011020 conn(0x7f69100785a0 msgr2=0x7f69100ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.435+0000 7f6918458700 1 -- 172.21.15.122:0/3531011020 shutdown_connections 2024-05-03T17:52:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.435+0000 7f6918458700 1 -- 172.21.15.122:0/3531011020 wait complete. 2024-05-03T17:52:20.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f6918458700 1 Processor -- start 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f6918458700 1 -- start start 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f6918458700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6910101eb0 0x7f691019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f6918458700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6910102870 0x7f691019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f6918458700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6910107b80 0x7f6910198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f6918458700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69101125a0 con 0x7f6910102870 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f6918458700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6910112420 con 0x7f6910101eb0 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f6918458700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6910112720 con 0x7f6910107b80 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f69159f3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6910102870 0x7f691019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f69159f3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6910102870 0x7f691019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36260/0 (socket says 172.21.15.122:36260) 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f69159f3700 1 -- 172.21.15.122:0/4048564368 learned_addr learned my addr 172.21.15.122:0/4048564368 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f69161f4700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6910101eb0 0x7f691019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f69169f5700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6910107b80 0x7f6910198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:20.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f69159f3700 1 -- 172.21.15.122:0/4048564368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6910107b80 msgr2=0x7f6910198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f69159f3700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6910107b80 0x7f6910198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f69159f3700 1 -- 172.21.15.122:0/4048564368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6910101eb0 msgr2=0x7f691019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f69159f3700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6910101eb0 0x7f691019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.436+0000 7f69159f3700 1 -- 172.21.15.122:0/4048564368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6910199060 con 0x7f6910102870 2024-05-03T17:52:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.437+0000 7f69159f3700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6910102870 0x7f691019ebf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f690000d950 tx=0x7f690000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.437+0000 7f69077fe700 1 -- 172.21.15.122:0/4048564368 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900016070 con 0x7f6910102870 2024-05-03T17:52:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.437+0000 7f6918458700 1 -- 172.21.15.122:0/4048564368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6910199350 con 0x7f6910102870 2024-05-03T17:52:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.437+0000 7f69077fe700 1 -- 172.21.15.122:0/4048564368 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6900009df0 con 0x7f6910102870 2024-05-03T17:52:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.437+0000 7f69077fe700 1 -- 172.21.15.122:0/4048564368 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690000f3c0 con 0x7f6910102870 2024-05-03T17:52:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.437+0000 7f6918458700 1 -- 172.21.15.122:0/4048564368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691010f360 con 0x7f6910102870 2024-05-03T17:52:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.438+0000 7f6918458700 1 -- 172.21.15.122:0/4048564368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6910103210 con 0x7f6910102870 2024-05-03T17:52:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.439+0000 7f69077fe700 1 -- 172.21.15.122:0/4048564368 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f6900019070 con 0x7f6910102870 2024-05-03T17:52:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.440+0000 7f69077fe700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f68fc0b2880 0x7f68fc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.440+0000 7f69161f4700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f68fc0b2880 0x7f68fc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.440+0000 7f69077fe700 1 -- 172.21.15.122:0/4048564368 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f69000e1a00 con 0x7f6910102870 2024-05-03T17:52:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.440+0000 7f69161f4700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f68fc0b2880 0x7f68fc0b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f690c005950 tx=0x7f690c00e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:20.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.444+0000 7f69077fe700 1 -- 172.21.15.122:0/4048564368 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f69000a9b20 con 0x7f6910102870 2024-05-03T17:52:20.537 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 17:52:20 smithi149 systemd[1]: Starting Ceph mds.b for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T17:52:20.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:20 smithi192 ceph-mon[33753]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:20.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.699+0000 7f6918458700 1 -- 172.21.15.122:0/4048564368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f691010fd90 con 0x7f6910102870 2024-05-03T17:52:20.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.700+0000 7f69077fe700 1 -- 172.21.15.122:0/4048564368 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9674 (secure 0 0 0) 0x7f69000a9420 con 0x7f6910102870 2024-05-03T17:52:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:52:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/555236113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":555236113},{"type":"v1","addr":"172.21.15.122:6839","nonce":555236113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1783292878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1783292878},{"type":"v1","addr":"172.21.15.122:6841","nonce":1783292878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24410,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/1429818440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1429818440},{"type":"v1","addr":"172.21.15.192:6839","nonce":1429818440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-03T17:52:20.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 -- 172.21.15.122:0/4048564368 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f68fc0b2880 msgr2=0x7f68fc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:20.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f68fc0b2880 0x7f68fc0b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f690c005950 tx=0x7f690c00e040 comp rx=0 tx=0).stop 2024-05-03T17:52:20.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 -- 172.21.15.122:0/4048564368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6910102870 msgr2=0x7f691019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:20.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6910102870 0x7f691019ebf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f690000d950 tx=0x7f690000dc60 comp rx=0 tx=0).stop 2024-05-03T17:52:20.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 -- 172.21.15.122:0/4048564368 shutdown_connections 2024-05-03T17:52:20.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f68fc0b2880 0x7f68fc0b4d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:20.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6910107b80 0x7f6910198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:20.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6910101eb0 0x7f691019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:20.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 --2- 172.21.15.122:0/4048564368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6910102870 0x7f691019ebf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:20.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 -- 172.21.15.122:0/4048564368 >> 172.21.15.122:0/4048564368 conn(0x7f69100785a0 msgr2=0x7f69100ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:20.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 -- 172.21.15.122:0/4048564368 shutdown_connections 2024-05-03T17:52:20.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:20.702+0000 7f6918458700 1 -- 172.21.15.122:0/4048564368 wait complete. 2024-05-03T17:52:20.699 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 12 2024-05-03T17:52:20.928 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 17:52:20 smithi149 podman[63617]: 2024-05-03 17:52:20.678483123 +0000 UTC m=+0.105073675 container create 4c322e9da4d48f97cd8011e836bf9d12b19ca94bd95c7bf9cf15d22227731732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-03T17:52:20.928 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 17:52:20 smithi149 podman[63617]: 2024-05-03 17:52:20.592104613 +0000 UTC m=+0.018695189 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T17:52:20.928 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 17:52:20 smithi149 podman[63617]: 2024-05-03 17:52:20.907380346 +0000 UTC m=+0.333970904 container init 4c322e9da4d48f97cd8011e836bf9d12b19ca94bd95c7bf9cf15d22227731732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-03T17:52:20.928 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 17:52:20 smithi149 podman[63617]: 2024-05-03 17:52:20.910097191 +0000 UTC m=+0.336687752 container start 4c322e9da4d48f97cd8011e836bf9d12b19ca94bd95c7bf9cf15d22227731732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-03T17:52:20.928 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 17:52:20 smithi149 ceph-mds[63636]: -- 172.21.15.149:0/1747684904 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560894a5da40 con 0x560894a30c00 2024-05-03T17:52:21.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:21 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/4048564368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:21.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:21 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.287 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 17:52:20 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b[63632]: starting mds.b at 2024-05-03T17:52:21.287 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 17:52:20 smithi149 bash[63617]: 4c322e9da4d48f97cd8011e836bf9d12b19ca94bd95c7bf9cf15d22227731732 2024-05-03T17:52:21.287 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 17:52:21 smithi149 systemd[1]: Started Ceph mds.b for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T17:52:21.359 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:21 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/4048564368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:21.359 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.360 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.360 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.360 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.360 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:21 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:21 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/4048564368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:21.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:21.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:21 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:22.272 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:22 smithi192 ceph-mon[33753]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:22.272 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:22 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.149:6838/1529632960,v1:172.21.15.149:6839/1529632960] up:boot 2024-05-03T17:52:22.272 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:22 smithi192 ceph-mon[33753]: fsmap 12 up:standby 2024-05-03T17:52:22.272 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:22 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T17:52:22.272 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:22 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:52:22.308 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:22 smithi149 ceph-mon[33409]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:22.308 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:22 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.149:6838/1529632960,v1:172.21.15.149:6839/1529632960] up:boot 2024-05-03T17:52:22.308 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:22 smithi149 ceph-mon[33409]: fsmap 12 up:standby 2024-05-03T17:52:22.308 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:22 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T17:52:22.308 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:22 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:52:22.347 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:22 smithi122 ceph-mon[29744]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:22.348 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:22 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.149:6838/1529632960,v1:172.21.15.149:6839/1529632960] up:boot 2024-05-03T17:52:22.348 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:22 smithi122 ceph-mon[29744]: fsmap 12 up:standby 2024-05-03T17:52:22.348 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:22 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T17:52:22.348 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:22 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:52:23.389 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:52:23.634 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:24.008 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:24.009 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.009 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.009 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.009 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.009 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.009 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.009 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.009 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:24.009 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:52:24.009 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.009 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:23 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:52:24.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:24.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:24.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:52:24.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:23 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:52:24.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:52:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:24.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:23 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:52:25.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.358+0000 7f552e197700 1 -- 172.21.15.122:0/1069724622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55281023a0 msgr2=0x7f5528102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:25.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.358+0000 7f552e197700 1 --2- 172.21.15.122:0/1069724622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55281023a0 0x7f5528102820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f551c009990 tx=0x7f551c009ca0 comp rx=0 tx=0).stop 2024-05-03T17:52:25.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.358+0000 7f552e197700 1 -- 172.21.15.122:0/1069724622 shutdown_connections 2024-05-03T17:52:25.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.358+0000 7f552e197700 1 --2- 172.21.15.122:0/1069724622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5528102d60 0x7f552810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:25.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.358+0000 7f552e197700 1 --2- 172.21.15.122:0/1069724622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5528108410 0x7f55281087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:25.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.358+0000 7f552e197700 1 --2- 172.21.15.122:0/1069724622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55281023a0 0x7f5528102820 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:25.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.358+0000 7f552e197700 1 -- 172.21.15.122:0/1069724622 >> 172.21.15.122:0/1069724622 conn(0x7f5528079350 msgr2=0x7f5528079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:25.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.358+0000 7f552e197700 1 -- 172.21.15.122:0/1069724622 shutdown_connections 2024-05-03T17:52:25.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.358+0000 7f552e197700 1 -- 172.21.15.122:0/1069724622 wait complete. 2024-05-03T17:52:25.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f552e197700 1 Processor -- start 2024-05-03T17:52:25.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f552e197700 1 -- start start 2024-05-03T17:52:25.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f552e197700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55281023a0 0x7f552819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:25.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f552e197700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5528102d60 0x7f552819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:25.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f552e197700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5528108410 0x7f5528199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:25.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f552e197700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5528112560 con 0x7f5528108410 2024-05-03T17:52:25.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f552e197700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f55281123e0 con 0x7f55281023a0 2024-05-03T17:52:25.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f552e197700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f55281126e0 con 0x7f5528102d60 2024-05-03T17:52:25.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f5527fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5528108410 0x7f5528199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f5526ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5528102d60 0x7f552819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f5526ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5528102d60 0x7f552819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.122:54246/0 (socket says 172.21.15.122:54246) 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f5527fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5528108410 0x7f5528199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36282/0 (socket says 172.21.15.122:36282) 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f5526ffd700 1 -- 172.21.15.122:0/857821296 learned_addr learned my addr 172.21.15.122:0/857821296 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.359+0000 7f55277fe700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55281023a0 0x7f552819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.360+0000 7f5526ffd700 1 -- 172.21.15.122:0/857821296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55281023a0 msgr2=0x7f552819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.360+0000 7f5526ffd700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55281023a0 0x7f552819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.360+0000 7f5526ffd700 1 -- 172.21.15.122:0/857821296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5528108410 msgr2=0x7f5528199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.360+0000 7f5526ffd700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5528108410 0x7f5528199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.360+0000 7f5526ffd700 1 -- 172.21.15.122:0/857821296 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552819a0a0 con 0x7f5528102d60 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.360+0000 7f5526ffd700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5528102d60 0x7f552819fcc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f551c000c00 tx=0x7f551c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.360+0000 7f5524ff9700 1 -- 172.21.15.122:0/857821296 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c027070 con 0x7f5528102d60 2024-05-03T17:52:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.360+0000 7f5524ff9700 1 -- 172.21.15.122:0/857821296 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f551c022470 con 0x7f5528102d60 2024-05-03T17:52:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.360+0000 7f552e197700 1 -- 172.21.15.122:0/857821296 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552819a330 con 0x7f5528102d60 2024-05-03T17:52:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.360+0000 7f5524ff9700 1 -- 172.21.15.122:0/857821296 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c017650 con 0x7f5528102d60 2024-05-03T17:52:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.360+0000 7f552e197700 1 -- 172.21.15.122:0/857821296 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5528103fb0 con 0x7f5528102d60 2024-05-03T17:52:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.361+0000 7f552e197700 1 -- 172.21.15.122:0/857821296 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5528103ac0 con 0x7f5528102d60 2024-05-03T17:52:25.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.362+0000 7f5524ff9700 1 -- 172.21.15.122:0/857821296 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f551c02a070 con 0x7f5528102d60 2024-05-03T17:52:25.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.363+0000 7f5524ff9700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f55100b2880 0x7f55100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:25.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.363+0000 7f55277fe700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f55100b2880 0x7f55100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:25.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.363+0000 7f5524ff9700 1 -- 172.21.15.122:0/857821296 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f551c0e2df0 con 0x7f5528102d60 2024-05-03T17:52:25.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.363+0000 7f55277fe700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f55100b2880 0x7f55100b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f552819aa80 tx=0x7f551800f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:25.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.367+0000 7f5524ff9700 1 -- 172.21.15.122:0/857821296 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f551c013070 con 0x7f5528102d60 2024-05-03T17:52:25.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.624+0000 7f552e197700 1 -- 172.21.15.122:0/857821296 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f552806a190 con 0x7f5528102d60 2024-05-03T17:52:25.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.625+0000 7f5524ff9700 1 -- 172.21.15.122:0/857821296 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10503 (secure 0 0 0) 0x7f552806a190 con 0x7f5528102d60 2024-05-03T17:52:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:52:25.621 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/555236113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":555236113},{"type":"v1","addr":"172.21.15.122:6839","nonce":555236113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1783292878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1783292878},{"type":"v1","addr":"172.21.15.122:6841","nonce":1783292878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24410,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/1529632960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1529632960},{"type":"v1","addr":"172.21.15.149:6839","nonce":1529632960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/1429818440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1429818440},{"type":"v1","addr":"172.21.15.192:6839","nonce":1429818440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-03T17:52:25.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 -- 172.21.15.122:0/857821296 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f55100b2880 msgr2=0x7f55100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f55100b2880 0x7f55100b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f552819aa80 tx=0x7f551800f820 comp rx=0 tx=0).stop 2024-05-03T17:52:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 -- 172.21.15.122:0/857821296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5528102d60 msgr2=0x7f552819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5528102d60 0x7f552819fcc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f551c000c00 tx=0x7f551c009580 comp rx=0 tx=0).stop 2024-05-03T17:52:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 -- 172.21.15.122:0/857821296 shutdown_connections 2024-05-03T17:52:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f55100b2880 0x7f55100b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5528108410 0x7f5528199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55281023a0 0x7f552819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 --2- 172.21.15.122:0/857821296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5528102d60 0x7f552819fcc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 -- 172.21.15.122:0/857821296 >> 172.21.15.122:0/857821296 conn(0x7f5528079350 msgr2=0x7f5528101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 -- 172.21.15.122:0/857821296 shutdown_connections 2024-05-03T17:52:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:25.627+0000 7f552e197700 1 -- 172.21.15.122:0/857821296 wait complete. 2024-05-03T17:52:25.624 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 13 2024-05-03T17:52:26.304 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-03T17:52:26.536 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:26 smithi122 ceph-mon[29744]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:26.537 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:26 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.537 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:26 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.537 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:26 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.537 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:26 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.537 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:26 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/857821296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:26.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:26 smithi149 ceph-mon[33409]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:26.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:26 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:26 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:26 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:26 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:26 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/857821296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:26.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:26 smithi192 ceph-mon[33753]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:26.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:26 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:26 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:26 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:26 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:26.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:26 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/857821296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:52:26.879 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:28.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.581+0000 7fd593628700 1 -- 172.21.15.122:0/2832997844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58c108560 msgr2=0x7fd58c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.581+0000 7fd593628700 1 --2- 172.21.15.122:0/2832997844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58c108560 0x7fd58c108940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd588009990 tx=0x7fd588009ca0 comp rx=0 tx=0).stop 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.581+0000 7fd593628700 1 -- 172.21.15.122:0/2832997844 shutdown_connections 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.581+0000 7fd593628700 1 --2- 172.21.15.122:0/2832997844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd58c102800 0x7fd58c110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.581+0000 7fd593628700 1 --2- 172.21.15.122:0/2832997844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58c108560 0x7fd58c108940 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.581+0000 7fd593628700 1 --2- 172.21.15.122:0/2832997844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c101e40 0x7fd58c1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.581+0000 7fd593628700 1 -- 172.21.15.122:0/2832997844 >> 172.21.15.122:0/2832997844 conn(0x7fd58c0776c0 msgr2=0x7fd58c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd593628700 1 -- 172.21.15.122:0/2832997844 shutdown_connections 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd593628700 1 -- 172.21.15.122:0/2832997844 wait complete. 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd593628700 1 Processor -- start 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd593628700 1 -- start start 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd593628700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd58c101e40 0x7fd58c103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd593628700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c102800 0x7fd58c103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:28.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd593628700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58c108560 0x7fd58c103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd593628700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c113650 con 0x7fd58c108560 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd593628700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1134d0 con 0x7fd58c101e40 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd593628700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1137d0 con 0x7fd58c102800 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd591bc5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58c108560 0x7fd58c103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd591bc5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58c108560 0x7fd58c103aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51116/0 (socket says 172.21.15.122:51116) 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.582+0000 7fd591bc5700 1 -- 172.21.15.122:0/916199245 learned_addr learned my addr 172.21.15.122:0/916199245 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd5913c4700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd58c101e40 0x7fd58c103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd590bc3700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c102800 0x7fd58c103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd591bc5700 1 -- 172.21.15.122:0/916199245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c102800 msgr2=0x7fd58c103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd591bc5700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c102800 0x7fd58c103560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd591bc5700 1 -- 172.21.15.122:0/916199245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd58c101e40 msgr2=0x7fd58c103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd591bc5700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd58c101e40 0x7fd58c103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd591bc5700 1 -- 172.21.15.122:0/916199245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c1a7ff0 con 0x7fd58c108560 2024-05-03T17:52:28.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd591bc5700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58c108560 0x7fd58c103aa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd584007d70 tx=0x7fd58400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:28.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd5827fc700 1 -- 172.21.15.122:0/916199245 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584016020 con 0x7fd58c108560 2024-05-03T17:52:28.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd5827fc700 1 -- 172.21.15.122:0/916199245 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd58400ad40 con 0x7fd58c108560 2024-05-03T17:52:28.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd593628700 1 -- 172.21.15.122:0/916199245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58c1a82e0 con 0x7fd58c108560 2024-05-03T17:52:28.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd5827fc700 1 -- 172.21.15.122:0/916199245 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5840115e0 con 0x7fd58c108560 2024-05-03T17:52:28.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.583+0000 7fd593628700 1 -- 172.21.15.122:0/916199245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58c1a87d0 con 0x7fd58c108560 2024-05-03T17:52:28.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.584+0000 7fd593628700 1 -- 172.21.15.122:0/916199245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58c06a190 con 0x7fd58c108560 2024-05-03T17:52:28.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.585+0000 7fd5827fc700 1 -- 172.21.15.122:0/916199245 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fd584019020 con 0x7fd58c108560 2024-05-03T17:52:28.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.586+0000 7fd5827fc700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd5780b2880 0x7fd5780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:28.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.586+0000 7fd5913c4700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd5780b2880 0x7fd5780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:28.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.587+0000 7fd5827fc700 1 -- 172.21.15.122:0/916199245 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fd5840e2150 con 0x7fd58c108560 2024-05-03T17:52:28.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.587+0000 7fd5913c4700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd5780b2880 0x7fd5780b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd588006010 tx=0x7fd58801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.590+0000 7fd5827fc700 1 -- 172.21.15.122:0/916199245 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd584020020 con 0x7fd58c108560 2024-05-03T17:52:28.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:28 smithi122 ceph-mon[29744]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:28 smithi149 ceph-mon[33409]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:28.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.806+0000 7fd593628700 1 -- 172.21.15.122:0/916199245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd58c1a8bd0 con 0x7fd58c108560 2024-05-03T17:52:28.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.806+0000 7fd5827fc700 1 -- 172.21.15.122:0/916199245 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7fd5840aa2f0 con 0x7fd58c108560 2024-05-03T17:52:28.801 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:52:28.801 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-05-03T17:52:28.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-05-03T17:52:28.801 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-05-03T17:52:28.802 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-05-03T17:52:28.802 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-05-03T17:52:28.802 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.808+0000 7fd593628700 1 -- 172.21.15.122:0/916199245 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd5780b2880 msgr2=0x7fd5780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.808+0000 7fd593628700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd5780b2880 0x7fd5780b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd588006010 tx=0x7fd58801a040 comp rx=0 tx=0).stop 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.808+0000 7fd593628700 1 -- 172.21.15.122:0/916199245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58c108560 msgr2=0x7fd58c103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.808+0000 7fd593628700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58c108560 0x7fd58c103aa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd584007d70 tx=0x7fd58400a340 comp rx=0 tx=0).stop 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.809+0000 7fd593628700 1 -- 172.21.15.122:0/916199245 shutdown_connections 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.809+0000 7fd593628700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd5780b2880 0x7fd5780b4d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.809+0000 7fd593628700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd58c108560 0x7fd58c103aa0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.809+0000 7fd593628700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd58c101e40 0x7fd58c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.809+0000 7fd593628700 1 --2- 172.21.15.122:0/916199245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c102800 0x7fd58c103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.809+0000 7fd593628700 1 -- 172.21.15.122:0/916199245 >> 172.21.15.122:0/916199245 conn(0x7fd58c0776c0 msgr2=0x7fd58c100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.809+0000 7fd593628700 1 -- 172.21.15.122:0/916199245 shutdown_connections 2024-05-03T17:52:28.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:28.809+0000 7fd593628700 1 -- 172.21.15.122:0/916199245 wait complete. 2024-05-03T17:52:28.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:28 smithi192 ceph-mon[33753]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:29.510 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-03T17:52:29.510 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-03T17:52:29.748 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:29.869 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:29 smithi122 ceph-mon[29744]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:29.870 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:29.870 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:29.870 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:29.870 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:52:29.870 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:29 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:29.870 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:29 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/916199245' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T17:52:30.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:29 smithi149 ceph-mon[33409]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:30.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:30.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:30.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:30.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:52:30.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:29 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:30.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:29 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/916199245' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T17:52:30.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:29 smithi192 ceph-mon[33753]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:30.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:30.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:30.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:52:30.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:52:30.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:29 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:52:30.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:29 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/916199245' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T17:52:31.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.428+0000 7f706f4ac700 1 -- 172.21.15.122:0/1426642062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7068108400 msgr2=0x7f70681087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:31.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.428+0000 7f706f4ac700 1 --2- 172.21.15.122:0/1426642062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7068108400 0x7f70681087e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7064009990 tx=0x7f7064009ca0 comp rx=0 tx=0).stop 2024-05-03T17:52:31.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.428+0000 7f706f4ac700 1 -- 172.21.15.122:0/1426642062 shutdown_connections 2024-05-03T17:52:31.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.428+0000 7f706f4ac700 1 --2- 172.21.15.122:0/1426642062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7068102d50 0x7f706810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:31.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.428+0000 7f706f4ac700 1 --2- 172.21.15.122:0/1426642062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7068108400 0x7f70681087e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:31.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.428+0000 7f706f4ac700 1 --2- 172.21.15.122:0/1426642062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7068102390 0x7f7068102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:31.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.428+0000 7f706f4ac700 1 -- 172.21.15.122:0/1426642062 >> 172.21.15.122:0/1426642062 conn(0x7f7068079310 msgr2=0x7f7068079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:31.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.428+0000 7f706f4ac700 1 -- 172.21.15.122:0/1426642062 shutdown_connections 2024-05-03T17:52:31.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.428+0000 7f706f4ac700 1 -- 172.21.15.122:0/1426642062 wait complete. 2024-05-03T17:52:31.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706f4ac700 1 Processor -- start 2024-05-03T17:52:31.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706f4ac700 1 -- start start 2024-05-03T17:52:31.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706f4ac700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7068102390 0x7f706819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:31.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706f4ac700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7068102d50 0x7f706819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:31.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706f4ac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7068108400 0x7f7068198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:31.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706f4ac700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7068112550 con 0x7f7068102390 2024-05-03T17:52:31.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706f4ac700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f70681123d0 con 0x7f7068102d50 2024-05-03T17:52:31.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706d248700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7068102390 0x7f706819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:31.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706f4ac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f70681126d0 con 0x7f7068108400 2024-05-03T17:52:31.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706d248700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7068102390 0x7f706819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51142/0 (socket says 172.21.15.122:51142) 2024-05-03T17:52:31.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706d248700 1 -- 172.21.15.122:0/293067014 learned_addr learned my addr 172.21.15.122:0/293067014 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:31.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.429+0000 7f706ca47700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7068102d50 0x7f706819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:31.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f706d248700 1 -- 172.21.15.122:0/293067014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7068108400 msgr2=0x7f7068198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:31.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f706da49700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7068108400 0x7f7068198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:31.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f706d248700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7068108400 0x7f7068198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:31.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f706d248700 1 -- 172.21.15.122:0/293067014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7068102d50 msgr2=0x7f706819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:31.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f706d248700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7068102d50 0x7f706819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:31.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f706d248700 1 -- 172.21.15.122:0/293067014 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7068198ff0 con 0x7f7068102390 2024-05-03T17:52:31.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f706d248700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7068102390 0x7f706819e6d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7064000c00 tx=0x7f70640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:31.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f705e7fc700 1 -- 172.21.15.122:0/293067014 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064027070 con 0x7f7068102390 2024-05-03T17:52:31.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f705e7fc700 1 -- 172.21.15.122:0/293067014 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7064003f20 con 0x7f7068102390 2024-05-03T17:52:31.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f706f4ac700 1 -- 172.21.15.122:0/293067014 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7068199280 con 0x7f7068102390 2024-05-03T17:52:31.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f705e7fc700 1 -- 172.21.15.122:0/293067014 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064022cf0 con 0x7f7068102390 2024-05-03T17:52:31.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.430+0000 7f706f4ac700 1 -- 172.21.15.122:0/293067014 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70681a55a0 con 0x7f7068102390 2024-05-03T17:52:31.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.431+0000 7f706f4ac700 1 -- 172.21.15.122:0/293067014 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7068103ab0 con 0x7f7068102390 2024-05-03T17:52:31.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.432+0000 7f705e7fc700 1 -- 172.21.15.122:0/293067014 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f706402a070 con 0x7f7068102390 2024-05-03T17:52:31.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.433+0000 7f705e7fc700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f70540b2880 0x7f70540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:31.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.433+0000 7f706ca47700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f70540b2880 0x7f70540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:31.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.433+0000 7f705e7fc700 1 -- 172.21.15.122:0/293067014 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f70640e2f00 con 0x7f7068102390 2024-05-03T17:52:31.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.433+0000 7f706ca47700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f70540b2880 0x7f70540b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7068079980 tx=0x7f705800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:31.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.436+0000 7f705e7fc700 1 -- 172.21.15.122:0/293067014 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7064013070 con 0x7f7068102390 2024-05-03T17:52:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.642+0000 7f706f4ac700 1 -- 172.21.15.122:0/293067014 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f706802e510 con 0x7f7068102390 2024-05-03T17:52:31.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.672+0000 7f705e7fc700 1 -- 172.21.15.122:0/293067014 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f70640ab150 con 0x7f7068102390 2024-05-03T17:52:31.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.676+0000 7f706f4ac700 1 -- 172.21.15.122:0/293067014 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f70540b2880 msgr2=0x7f70540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:31.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.676+0000 7f706f4ac700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f70540b2880 0x7f70540b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7068079980 tx=0x7f705800b410 comp rx=0 tx=0).stop 2024-05-03T17:52:31.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.676+0000 7f706f4ac700 1 -- 172.21.15.122:0/293067014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7068102390 msgr2=0x7f706819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:31.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.676+0000 7f706f4ac700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7068102390 0x7f706819e6d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7064000c00 tx=0x7f70640047b0 comp rx=0 tx=0).stop 2024-05-03T17:52:31.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.676+0000 7f706f4ac700 1 -- 172.21.15.122:0/293067014 shutdown_connections 2024-05-03T17:52:31.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.676+0000 7f706f4ac700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f70540b2880 0x7f70540b4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:31.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.676+0000 7f706f4ac700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7068108400 0x7f7068198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:31.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.676+0000 7f706f4ac700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7068102390 0x7f706819e6d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:31.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.676+0000 7f706f4ac700 1 --2- 172.21.15.122:0/293067014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7068102d50 0x7f706819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:31.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.676+0000 7f706f4ac700 1 -- 172.21.15.122:0/293067014 >> 172.21.15.122:0/293067014 conn(0x7f7068079310 msgr2=0x7f70681010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:31.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.677+0000 7f706f4ac700 1 -- 172.21.15.122:0/293067014 shutdown_connections 2024-05-03T17:52:31.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:31.677+0000 7f706f4ac700 1 -- 172.21.15.122:0/293067014 wait complete. 2024-05-03T17:52:31.674 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-05-03T17:52:31.932 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:31 smithi192 ceph-mon[33753]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:31.932 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:31 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/293067014' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T17:52:31.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:31 smithi122 ceph-mon[29744]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:31.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:31 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/293067014' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T17:52:32.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:31 smithi149 ceph-mon[33409]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:32.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:31 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/293067014' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T17:52:32.354 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-03T17:52:32.596 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:32.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:32 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/293067014' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T17:52:32.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:32 smithi122 ceph-mon[29744]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T17:52:33.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:32 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/293067014' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T17:52:33.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:32 smithi149 ceph-mon[33409]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T17:52:33.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:32 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/293067014' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T17:52:33.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:32 smithi192 ceph-mon[33753]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T17:52:33.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:33 smithi122 ceph-mon[29744]: pgmap v278: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:33.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:33 smithi122 ceph-mon[29744]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T17:52:33.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:33 smithi122 ceph-mon[29744]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T17:52:34.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:33 smithi149 ceph-mon[33409]: pgmap v278: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:34.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:33 smithi149 ceph-mon[33409]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T17:52:34.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:33 smithi149 ceph-mon[33409]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T17:52:34.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:33 smithi192 ceph-mon[33753]: pgmap v278: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:34.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:33 smithi192 ceph-mon[33753]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T17:52:34.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:33 smithi192 ceph-mon[33753]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T17:52:34.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.284+0000 7f37665b7700 1 -- 172.21.15.122:0/496588257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37601076e0 msgr2=0x7f376010f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:34.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.284+0000 7f37665b7700 1 --2- 172.21.15.122:0/496588257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37601076e0 0x7f376010f2b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f375800b0d0 tx=0x7f375800b3e0 comp rx=0 tx=0).stop 2024-05-03T17:52:34.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.284+0000 7f37665b7700 1 -- 172.21.15.122:0/496588257 shutdown_connections 2024-05-03T17:52:34.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.284+0000 7f37665b7700 1 --2- 172.21.15.122:0/496588257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37601076e0 0x7f376010f2b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:34.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.284+0000 7f37665b7700 1 --2- 172.21.15.122:0/496588257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3760102690 0x7f3760102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:34.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.284+0000 7f37665b7700 1 --2- 172.21.15.122:0/496588257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3760103040 0x7f3760107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:34.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.284+0000 7f37665b7700 1 -- 172.21.15.122:0/496588257 >> 172.21.15.122:0/496588257 conn(0x7f37600783f0 msgr2=0x7f3760078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:34.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.284+0000 7f37665b7700 1 -- 172.21.15.122:0/496588257 shutdown_connections 2024-05-03T17:52:34.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.284+0000 7f37665b7700 1 -- 172.21.15.122:0/496588257 wait complete. 2024-05-03T17:52:34.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.284+0000 7f37665b7700 1 Processor -- start 2024-05-03T17:52:34.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f37665b7700 1 -- start start 2024-05-03T17:52:34.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f37665b7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3760102690 0x7f3760198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:34.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f37665b7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3760103040 0x7f3760198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:34.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f37665b7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37601076e0 0x7f37601991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:34.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f37665b7700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3760112580 con 0x7f37601076e0 2024-05-03T17:52:34.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f37665b7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3760112400 con 0x7f3760102690 2024-05-03T17:52:34.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f37665b7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3760112700 con 0x7f3760103040 2024-05-03T17:52:34.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f3764b54700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37601076e0 0x7f37601991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:34.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f3764b54700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37601076e0 0x7f37601991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51154/0 (socket says 172.21.15.122:51154) 2024-05-03T17:52:34.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f3764b54700 1 -- 172.21.15.122:0/752221548 learned_addr learned my addr 172.21.15.122:0/752221548 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:34.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f375ffff700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3760102690 0x7f3760198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f375f7fe700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3760103040 0x7f3760198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f3764b54700 1 -- 172.21.15.122:0/752221548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3760103040 msgr2=0x7f3760198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f3764b54700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3760103040 0x7f3760198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f3764b54700 1 -- 172.21.15.122:0/752221548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3760102690 msgr2=0x7f3760198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f3764b54700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3760102690 0x7f3760198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f3764b54700 1 -- 172.21.15.122:0/752221548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37601a7fe0 con 0x7f37601076e0 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.285+0000 7f375ffff700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3760102690 0x7f3760198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.286+0000 7f3764b54700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37601076e0 0x7f37601991f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3758009850 tx=0x7f3758007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.286+0000 7f375d7fa700 1 -- 172.21.15.122:0/752221548 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375801f070 con 0x7f37601076e0 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.286+0000 7f375d7fa700 1 -- 172.21.15.122:0/752221548 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3758004c50 con 0x7f37601076e0 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.286+0000 7f375d7fa700 1 -- 172.21.15.122:0/752221548 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375801b690 con 0x7f37601076e0 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.286+0000 7f37665b7700 1 -- 172.21.15.122:0/752221548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37601a8270 con 0x7f37601076e0 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.286+0000 7f37665b7700 1 -- 172.21.15.122:0/752221548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37601a86c0 con 0x7f37601076e0 2024-05-03T17:52:34.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.287+0000 7f37665b7700 1 -- 172.21.15.122:0/752221548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376006a190 con 0x7f37601076e0 2024-05-03T17:52:34.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.288+0000 7f375d7fa700 1 -- 172.21.15.122:0/752221548 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3758022070 con 0x7f37601076e0 2024-05-03T17:52:34.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.289+0000 7f375d7fa700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f37480b2880 0x7f37480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:34.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.289+0000 7f375ffff700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f37480b2880 0x7f37480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:34.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.289+0000 7f375d7fa700 1 -- 172.21.15.122:0/752221548 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7497+0+0 (secure 0 0 0) 0x7f375800f020 con 0x7f37601076e0 2024-05-03T17:52:34.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.289+0000 7f375ffff700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f37480b2880 0x7f37480b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3750007fd0 tx=0x7f37500073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:34.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.292+0000 7f375d7fa700 1 -- 172.21.15.122:0/752221548 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3758029020 con 0x7f37601076e0 2024-05-03T17:52:34.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.498+0000 7f37665b7700 1 -- 172.21.15.122:0/752221548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f376002e510 con 0x7f37601076e0 2024-05-03T17:52:34.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.695+0000 7f375d7fa700 1 -- 172.21.15.122:0/752221548 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v74) ==== 170+0+0 (secure 0 0 0) 0x7f3758009cb0 con 0x7f37601076e0 2024-05-03T17:52:34.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 -- 172.21.15.122:0/752221548 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f37480b2880 msgr2=0x7f37480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:34.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f37480b2880 0x7f37480b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3750007fd0 tx=0x7f37500073d0 comp rx=0 tx=0).stop 2024-05-03T17:52:34.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 -- 172.21.15.122:0/752221548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37601076e0 msgr2=0x7f37601991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:34.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37601076e0 0x7f37601991f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3758009850 tx=0x7f3758007a20 comp rx=0 tx=0).stop 2024-05-03T17:52:34.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 -- 172.21.15.122:0/752221548 shutdown_connections 2024-05-03T17:52:34.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f37480b2880 0x7f37480b4d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:34.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37601076e0 0x7f37601991f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:34.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3760102690 0x7f3760198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:34.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 --2- 172.21.15.122:0/752221548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3760103040 0x7f3760198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:34.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 -- 172.21.15.122:0/752221548 >> 172.21.15.122:0/752221548 conn(0x7f37600783f0 msgr2=0x7f3760100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:34.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 -- 172.21.15.122:0/752221548 shutdown_connections 2024-05-03T17:52:34.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:34.698+0000 7f37665b7700 1 -- 172.21.15.122:0/752221548 wait complete. 2024-05-03T17:52:34.694 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-05-03T17:52:34.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:34 smithi122 ceph-mon[29744]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T17:52:34.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:34 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/752221548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T17:52:35.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:34 smithi149 ceph-mon[33409]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T17:52:35.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:34 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/752221548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T17:52:35.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:34 smithi192 ceph-mon[33753]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T17:52:35.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:34 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/752221548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T17:52:35.401 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-03T17:52:35.638 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:35.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:35 smithi122 ceph-mon[29744]: pgmap v281: 33 pgs: 23 active+clean, 3 creating+peering, 7 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:35.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:35 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/752221548' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T17:52:35.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:35 smithi122 ceph-mon[29744]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T17:52:36.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:35 smithi149 ceph-mon[33409]: pgmap v281: 33 pgs: 23 active+clean, 3 creating+peering, 7 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:36.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:35 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/752221548' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T17:52:36.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:35 smithi149 ceph-mon[33409]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T17:52:36.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:35 smithi192 ceph-mon[33753]: pgmap v281: 33 pgs: 23 active+clean, 3 creating+peering, 7 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:36.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:35 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/752221548' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T17:52:36.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:35 smithi192 ceph-mon[33753]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T17:52:36.804 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:36 smithi122 ceph-mon[29744]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T17:52:37.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:36 smithi149 ceph-mon[33409]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T17:52:37.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:36 smithi192 ceph-mon[33753]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T17:52:37.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.356+0000 7f590fee6700 1 -- 172.21.15.122:0/2606099491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59080756d0 msgr2=0x7f5908075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:37.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.356+0000 7f590fee6700 1 --2- 172.21.15.122:0/2606099491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59080756d0 0x7f5908075ab0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f590400b0a0 tx=0x7f590400b3b0 comp rx=0 tx=0).stop 2024-05-03T17:52:37.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.356+0000 7f590fee6700 1 -- 172.21.15.122:0/2606099491 shutdown_connections 2024-05-03T17:52:37.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.356+0000 7f590fee6700 1 --2- 172.21.15.122:0/2606099491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f590810fea0 0x7f5908112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:37.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.356+0000 7f590fee6700 1 --2- 172.21.15.122:0/2606099491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59080756d0 0x7f5908075ab0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:37.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.356+0000 7f590fee6700 1 --2- 172.21.15.122:0/2606099491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5908075ff0 0x7f590810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:37.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.356+0000 7f590fee6700 1 -- 172.21.15.122:0/2606099491 >> 172.21.15.122:0/2606099491 conn(0x7f590807a650 msgr2=0x7f590807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:37.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.356+0000 7f590fee6700 1 -- 172.21.15.122:0/2606099491 shutdown_connections 2024-05-03T17:52:37.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.356+0000 7f590fee6700 1 -- 172.21.15.122:0/2606099491 wait complete. 2024-05-03T17:52:37.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.356+0000 7f590fee6700 1 Processor -- start 2024-05-03T17:52:37.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.356+0000 7f590fee6700 1 -- start start 2024-05-03T17:52:37.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590fee6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59080756d0 0x7f590819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:37.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590fee6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5908075ff0 0x7f590819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:37.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590fee6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f590810fea0 0x7f59081a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:37.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590fee6700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59081159a0 con 0x7f590810fea0 2024-05-03T17:52:37.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590fee6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5908115820 con 0x7f5908075ff0 2024-05-03T17:52:37.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590fee6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5908115b20 con 0x7f59080756d0 2024-05-03T17:52:37.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590e483700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f590810fea0 0x7f59081a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590e483700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f590810fea0 0x7f59081a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51178/0 (socket says 172.21.15.122:51178) 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590e483700 1 -- 172.21.15.122:0/28591332 learned_addr learned my addr 172.21.15.122:0/28591332 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590dc82700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59080756d0 0x7f590819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590d481700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5908075ff0 0x7f590819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590e483700 1 -- 172.21.15.122:0/28591332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59080756d0 msgr2=0x7f590819bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590e483700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59080756d0 0x7f590819bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590e483700 1 -- 172.21.15.122:0/28591332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5908075ff0 msgr2=0x7f590819c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590e483700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5908075ff0 0x7f590819c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590e483700 1 -- 172.21.15.122:0/28591332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59081a0f20 con 0x7f590810fea0 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590d481700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5908075ff0 0x7f590819c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590e483700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f590810fea0 0x7f59081a07e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f590000b410 tx=0x7f5900009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f58feffd700 1 -- 172.21.15.122:0/28591332 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900015020 con 0x7f590810fea0 2024-05-03T17:52:37.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f58feffd700 1 -- 172.21.15.122:0/28591332 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5900011ce0 con 0x7f590810fea0 2024-05-03T17:52:37.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.357+0000 7f590fee6700 1 -- 172.21.15.122:0/28591332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59081a1210 con 0x7f590810fea0 2024-05-03T17:52:37.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.358+0000 7f58feffd700 1 -- 172.21.15.122:0/28591332 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900010c10 con 0x7f590810fea0 2024-05-03T17:52:37.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.358+0000 7f590fee6700 1 -- 172.21.15.122:0/28591332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59081a8a90 con 0x7f590810fea0 2024-05-03T17:52:37.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.359+0000 7f590fee6700 1 -- 172.21.15.122:0/28591332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f590806a190 con 0x7f590810fea0 2024-05-03T17:52:37.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.360+0000 7f58feffd700 1 -- 172.21.15.122:0/28591332 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f5900018020 con 0x7f590810fea0 2024-05-03T17:52:37.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.360+0000 7f58feffd700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f58f40b2880 0x7f58f40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:37.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.361+0000 7f590dc82700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f58f40b2880 0x7f58f40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:37.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.361+0000 7f58feffd700 1 -- 172.21.15.122:0/28591332 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7879+0+0 (secure 0 0 0) 0x7f59000e1680 con 0x7f590810fea0 2024-05-03T17:52:37.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.361+0000 7f590dc82700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f58f40b2880 0x7f58f40b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5904000f80 tx=0x7f59040060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:37.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.366+0000 7f58feffd700 1 -- 172.21.15.122:0/28591332 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f590001f020 con 0x7f590810fea0 2024-05-03T17:52:37.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.615+0000 7f590fee6700 1 -- 172.21.15.122:0/28591332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f590802e510 con 0x7f590810fea0 2024-05-03T17:52:37.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.711+0000 7f58feffd700 1 -- 172.21.15.122:0/28591332 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f59000a9550 con 0x7f590810fea0 2024-05-03T17:52:37.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 -- 172.21.15.122:0/28591332 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f58f40b2880 msgr2=0x7f58f40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:37.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f58f40b2880 0x7f58f40b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5904000f80 tx=0x7f59040060c0 comp rx=0 tx=0).stop 2024-05-03T17:52:37.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 -- 172.21.15.122:0/28591332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f590810fea0 msgr2=0x7f59081a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:37.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f590810fea0 0x7f59081a07e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f590000b410 tx=0x7f5900009f90 comp rx=0 tx=0).stop 2024-05-03T17:52:37.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 -- 172.21.15.122:0/28591332 shutdown_connections 2024-05-03T17:52:37.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f58f40b2880 0x7f58f40b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:37.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f590810fea0 0x7f59081a07e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:37.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59080756d0 0x7f590819bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:37.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 --2- 172.21.15.122:0/28591332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5908075ff0 0x7f590819c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:37.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 -- 172.21.15.122:0/28591332 >> 172.21.15.122:0/28591332 conn(0x7f590807a650 msgr2=0x7f59081007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:37.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 -- 172.21.15.122:0/28591332 shutdown_connections 2024-05-03T17:52:37.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:37.713+0000 7f590fee6700 1 -- 172.21.15.122:0/28591332 wait complete. 2024-05-03T17:52:37.710 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 2 and data pool 3 2024-05-03T17:52:37.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:37 smithi122 ceph-mon[29744]: pgmap v284: 97 pgs: 38 active+clean, 7 creating+peering, 52 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:37.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:37 smithi122 ceph-mon[29744]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T17:52:37.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:37 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/28591332' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T17:52:37.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:37 smithi122 ceph-mon[29744]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-03T17:52:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:37 smithi149 ceph-mon[33409]: pgmap v284: 97 pgs: 38 active+clean, 7 creating+peering, 52 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:37 smithi149 ceph-mon[33409]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T17:52:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:37 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/28591332' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T17:52:38.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:37 smithi149 ceph-mon[33409]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-03T17:52:38.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:37 smithi192 ceph-mon[33753]: pgmap v284: 97 pgs: 38 active+clean, 7 creating+peering, 52 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T17:52:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:37 smithi192 ceph-mon[33753]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T17:52:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:37 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/28591332' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T17:52:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:37 smithi192 ceph-mon[33753]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-03T17:52:38.494 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T17:52:38.738 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:38.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:38 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/28591332' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T17:52:38.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:38 smithi122 ceph-mon[29744]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T17:52:38.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:38 smithi122 ceph-mon[29744]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-03T17:52:38.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:38 smithi122 ceph-mon[29744]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-03T17:52:39.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:38 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/28591332' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T17:52:39.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:38 smithi149 ceph-mon[33409]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T17:52:39.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:38 smithi149 ceph-mon[33409]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-03T17:52:39.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:38 smithi149 ceph-mon[33409]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-03T17:52:39.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:38 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/28591332' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T17:52:39.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:38 smithi192 ceph-mon[33753]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T17:52:39.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:38 smithi192 ceph-mon[33753]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-03T17:52:39.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:38 smithi192 ceph-mon[33753]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-03T17:52:39.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:39 smithi122 ceph-mon[29744]: pgmap v287: 97 pgs: 67 active+clean, 4 creating+peering, 26 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-03T17:52:39.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:39 smithi122 ceph-mon[29744]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T17:52:39.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:39 smithi122 ceph-mon[29744]: Cluster is now healthy 2024-05-03T17:52:39.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:39 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] up:active 2024-05-03T17:52:39.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:39 smithi122 ceph-mon[29744]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:39.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:39 smithi122 ceph-mon[29744]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:40.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:39 smithi149 ceph-mon[33409]: pgmap v287: 97 pgs: 67 active+clean, 4 creating+peering, 26 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-03T17:52:40.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:39 smithi149 ceph-mon[33409]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T17:52:40.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:39 smithi149 ceph-mon[33409]: Cluster is now healthy 2024-05-03T17:52:40.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:39 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] up:active 2024-05-03T17:52:40.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:39 smithi149 ceph-mon[33409]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:40.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:39 smithi149 ceph-mon[33409]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:40.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:39 smithi192 ceph-mon[33753]: pgmap v287: 97 pgs: 67 active+clean, 4 creating+peering, 26 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-03T17:52:40.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:39 smithi192 ceph-mon[33753]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T17:52:40.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:39 smithi192 ceph-mon[33753]: Cluster is now healthy 2024-05-03T17:52:40.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:39 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] up:active 2024-05-03T17:52:40.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:39 smithi192 ceph-mon[33753]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:40.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:39 smithi192 ceph-mon[33753]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:40.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.497+0000 7fcd263ec700 1 -- 172.21.15.122:0/3118450501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20103520 msgr2=0x7fcd2010f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:40.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.497+0000 7fcd263ec700 1 --2- 172.21.15.122:0/3118450501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20103520 0x7fcd2010f230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800b0d0 tx=0x7fcd1800b3e0 comp rx=0 tx=0).stop 2024-05-03T17:52:40.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.497+0000 7fcd263ec700 1 -- 172.21.15.122:0/3118450501 shutdown_connections 2024-05-03T17:52:40.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.497+0000 7fcd263ec700 1 --2- 172.21.15.122:0/3118450501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20103520 0x7fcd2010f230 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.497+0000 7fcd263ec700 1 --2- 172.21.15.122:0/3118450501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd20107420 0x7fcd20107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.497+0000 7fcd263ec700 1 --2- 172.21.15.122:0/3118450501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd20102b60 0x7fcd20102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.497+0000 7fcd263ec700 1 -- 172.21.15.122:0/3118450501 >> 172.21.15.122:0/3118450501 conn(0x7fcd200789a0 msgr2=0x7fcd200ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.497+0000 7fcd263ec700 1 -- 172.21.15.122:0/3118450501 shutdown_connections 2024-05-03T17:52:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.498+0000 7fcd263ec700 1 -- 172.21.15.122:0/3118450501 wait complete. 2024-05-03T17:52:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.498+0000 7fcd263ec700 1 Processor -- start 2024-05-03T17:52:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.498+0000 7fcd263ec700 1 -- start start 2024-05-03T17:52:40.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.498+0000 7fcd263ec700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd20102b60 0x7fcd2019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:40.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.498+0000 7fcd263ec700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd20103520 0x7fcd2019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:40.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd263ec700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20107420 0x7fcd201a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:40.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd263ec700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20112560 con 0x7fcd20107420 2024-05-03T17:52:40.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd263ec700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201123e0 con 0x7fcd20103520 2024-05-03T17:52:40.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd263ec700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201126e0 con 0x7fcd20102b60 2024-05-03T17:52:40.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd24989700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20107420 0x7fcd201a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:40.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd24989700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20107420 0x7fcd201a5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37734/0 (socket says 172.21.15.122:37734) 2024-05-03T17:52:40.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd24989700 1 -- 172.21.15.122:0/2246605358 learned_addr learned my addr 172.21.15.122:0/2246605358 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:40.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd1f7fe700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd20103520 0x7fcd2019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:40.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd1ffff700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd20102b60 0x7fcd2019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd24989700 1 -- 172.21.15.122:0/2246605358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd20102b60 msgr2=0x7fcd2019a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd24989700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd20102b60 0x7fcd2019a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd24989700 1 -- 172.21.15.122:0/2246605358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd20103520 msgr2=0x7fcd2019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd24989700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd20103520 0x7fcd2019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd24989700 1 -- 172.21.15.122:0/2246605358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd201a5e40 con 0x7fcd20107420 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd1ffff700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd20102b60 0x7fcd2019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd24989700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20107420 0x7fcd201a5900 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcd18007a20 tx=0x7fcd18007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd1d7fa700 1 -- 172.21.15.122:0/2246605358 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18021020 con 0x7fcd20107420 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd1d7fa700 1 -- 172.21.15.122:0/2246605358 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd18004c50 con 0x7fcd20107420 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd263ec700 1 -- 172.21.15.122:0/2246605358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd201a6010 con 0x7fcd20107420 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd1d7fa700 1 -- 172.21.15.122:0/2246605358 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd180052b0 con 0x7fcd20107420 2024-05-03T17:52:40.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.499+0000 7fcd263ec700 1 -- 172.21.15.122:0/2246605358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd201a64c0 con 0x7fcd20107420 2024-05-03T17:52:40.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.500+0000 7fcd263ec700 1 -- 172.21.15.122:0/2246605358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2006a190 con 0x7fcd20107420 2024-05-03T17:52:40.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.502+0000 7fcd1d7fa700 1 -- 172.21.15.122:0/2246605358 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fcd18024020 con 0x7fcd20107420 2024-05-03T17:52:40.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.503+0000 7fcd1d7fa700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcd080b2900 0x7fcd080b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:40.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.503+0000 7fcd1ffff700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcd080b2900 0x7fcd080b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:40.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.503+0000 7fcd1d7fa700 1 -- 172.21.15.122:0/2246605358 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fcd1800f020 con 0x7fcd20107420 2024-05-03T17:52:40.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.503+0000 7fcd1ffff700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcd080b2900 0x7fcd080b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009c90 tx=0x7fcd10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:40.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.507+0000 7fcd1d7fa700 1 -- 172.21.15.122:0/2246605358 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcd180ab7d0 con 0x7fcd20107420 2024-05-03T17:52:40.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.720+0000 7fcd263ec700 1 -- 172.21.15.122:0/2246605358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd201a6880 con 0x7fcd20107420 2024-05-03T17:52:40.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.721+0000 7fcd1d7fa700 1 -- 172.21.15.122:0/2246605358 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22283 (secure 0 0 0) 0x7fcd180ab0d0 con 0x7fcd20107420 2024-05-03T17:52:40.716 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:52:40.717 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":77,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","created":"2024-05-03T17:44:17.415571+0000","modified":"2024-05-03T17:52:37.616764+0000","last_up_change":"2024-05-03T17:51:28.839117+0000","last_in_change":"2024-05-03T17:51:07.954070+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T17:47:40.539967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T17:52:31.646919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T17:52:34.503067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4b08628-0a29-44aa-b8b9-ed9657f12be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6803","nonce":1329364610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6805","nonce":1329364610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6809","nonce":1329364610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6807","nonce":1329364610}]},"public_addr":"172.21.15.122:6803/1329364610","cluster_addr":"172.21.15.122:6805/1329364610","heartbeat_back_addr":"172.21.15.122:6809/1329364610","heartbeat_front_addr":"172.21.15.122:6807/1329364610","state":["exists","up"]},{"osd":1,"uuid":"ba550787-ce76-4944-8ebd-97e19590bb3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6801","nonce":1402463302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6803","nonce":1402463302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6807","nonce":1402463302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6805","nonce":1402463302}]},"public_addr":"172.21.15.149:6801/1402463302","cluster_addr":"172.21.15.149:6803/1402463302","heartbeat_back_addr":"172.21.15.149:6807/1402463302","heartbeat_front_addr":"172.21.15.149:6805/1402463302","state":["exists","up"]},{"osd":2,"uuid":"35ae7c47-d710-47c5-b0ec-91fa12b5be07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6801","nonce":3757191110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6803","nonce":3757191110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6807","nonce":3757191110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6805","nonce":3757191110}]},"public_addr":"172.21.15.192:6801/3757191110","cluster_addr":"172.21.15.192:6803/3757191110","heartbeat_back_addr":"172.21.15.192:6807/3757191110","heartbeat_front_addr":"172.21.15.192:6805/3757191110","state":["exists","up"]},{"osd":3,"uuid":"9973c0c1-ae73-497a-845e-664a20666df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6811","nonce":2359143764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6813","nonce":2359143764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6817","nonce":2359143764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6815","nonce":2359143764}]},"public_addr":"172.21.15.122:6811/2359143764","cluster_addr":"172.21.15.122:6813/2359143764","heartbeat_back_addr":"172.21.15.122:6817/2359143764","heartbeat_front_addr":"172.21.15.122:6815/2359143764","state":["exists","up"]},{"osd":4,"uuid":"da71dc18-94b9-4f7d-a061-7cc7b77db282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3627037532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3627037532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3627037532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3627037532}]},"public_addr":"172.21.15.149:6809/3627037532","cluster_addr":"172.21.15.149:6811/3627037532","heartbeat_back_addr":"172.21.15.149:6815/3627037532","heartbeat_front_addr":"172.21.15.149:6813/3627037532","state":["exists","up"]},{"osd":5,"uuid":"bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6809","nonce":455779485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6811","nonce":455779485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6815","nonce":455779485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6813","nonce":455779485}]},"public_addr":"172.21.15.192:6809/455779485","cluster_addr":"172.21.15.192:6811/455779485","heartbeat_back_addr":"172.21.15.192:6815/455779485","heartbeat_front_addr":"172.21.15.192:6813/455779485","state":["exists","up"]},{"osd":6,"uuid":"4150bfe5-a0e4-4bae-914a-75598f017519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6819","nonce":3419421998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6821","nonce":3419421998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6825","nonce":3419421998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6823","nonce":3419421998}]},"public_addr":"172.21.15.122:6819/3419421998","cluster_addr":"172.21.15.122:6821/3419421998","heartbeat_back_addr":"172.21.15.122:6825/3419421998","heartbeat_front_addr":"172.21.15.122:6823/3419421998","state":["exists","up"]},{"osd":7,"uuid":"71e01812-e511-44a0-a143-2afca510ae8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6817","nonce":139747549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6819","nonce":139747549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6823","nonce":139747549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6821","nonce":139747549}]},"public_addr":"172.21.15.149:6817/139747549","cluster_addr":"172.21.15.149:6819/139747549","heartbeat_back_addr":"172.21.15.149:6823/139747549","heartbeat_front_addr":"172.21.15.149:6821/139747549","state":["exists","up"]},{"osd":8,"uuid":"5294516d-fac6-41d0-a635-6bc1d7eca585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6817","nonce":2080816744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6819","nonce":2080816744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6823","nonce":2080816744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6821","nonce":2080816744}]},"public_addr":"172.21.15.192:6817/2080816744","cluster_addr":"172.21.15.192:6819/2080816744","heartbeat_back_addr":"172.21.15.192:6823/2080816744","heartbeat_front_addr":"172.21.15.192:6821/2080816744","state":["exists","up"]},{"osd":9,"uuid":"f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6827","nonce":1773691421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6829","nonce":1773691421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6833","nonce":1773691421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6831","nonce":1773691421}]},"public_addr":"172.21.15.122:6827/1773691421","cluster_addr":"172.21.15.122:6829/1773691421","heartbeat_back_addr":"172.21.15.122:6833/1773691421","heartbeat_front_addr":"172.21.15.122:6831/1773691421","state":["exists","up"]},{"osd":10,"uuid":"af88e50f-98f1-460d-856b-a920dd2ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6825","nonce":1954186137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6827","nonce":1954186137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6831","nonce":1954186137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6829","nonce":1954186137}]},"public_addr":"172.21.15.149:6825/1954186137","cluster_addr":"172.21.15.149:6827/1954186137","heartbeat_back_addr":"172.21.15.149:6831/1954186137","heartbeat_front_addr":"172.21.15.149:6829/1954186137","state":["exists","up"]},{"osd":11,"uuid":"b54c1b4d-cdb5-42d9-979a-47f9a870b5cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6825","nonce":2098474209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6827","nonce":2098474209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6831","nonce":2098474209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6829","nonce":2098474209}]},"public_addr":"172.21.15.192:6825/2098474209","cluster_addr":"172.21.15.192:6827/2098474209","heartbeat_back_addr":"172.21.15.192:6831/2098474209","heartbeat_front_addr":"172.21.15.192:6829/2098474209","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:46:50.049769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:13.518485+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:37.473023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:01.809640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:26.648514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:51.268056+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:16.294834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:42.079065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:07.609271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:33.885127+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:59.982708+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:51:26.461369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/2089907395":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6801/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:0/2001860625":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/1888284988":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/256574472":"2024-05-04T17:44:52.244049+0000","172.21.15.122:6801/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/64546794":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/584846864":"2024-05-04T17:45:18.454406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.723+0000 7fcd263ec700 1 -- 172.21.15.122:0/2246605358 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcd080b2900 msgr2=0x7fcd080b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.723+0000 7fcd263ec700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcd080b2900 0x7fcd080b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009c90 tx=0x7fcd10009260 comp rx=0 tx=0).stop 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.723+0000 7fcd263ec700 1 -- 172.21.15.122:0/2246605358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20107420 msgr2=0x7fcd201a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.723+0000 7fcd263ec700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20107420 0x7fcd201a5900 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcd18007a20 tx=0x7fcd18007a50 comp rx=0 tx=0).stop 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.723+0000 7fcd263ec700 1 -- 172.21.15.122:0/2246605358 shutdown_connections 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.723+0000 7fcd263ec700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcd080b2900 0x7fcd080b4dc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.723+0000 7fcd263ec700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20107420 0x7fcd201a5900 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.723+0000 7fcd263ec700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd20102b60 0x7fcd2019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.723+0000 7fcd263ec700 1 --2- 172.21.15.122:0/2246605358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd20103520 0x7fcd2019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.723+0000 7fcd263ec700 1 -- 172.21.15.122:0/2246605358 >> 172.21.15.122:0/2246605358 conn(0x7fcd200789a0 msgr2=0x7fcd20104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.724+0000 7fcd263ec700 1 -- 172.21.15.122:0/2246605358 shutdown_connections 2024-05-03T17:52:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:40.724+0000 7fcd263ec700 1 -- 172.21.15.122:0/2246605358 wait complete. 2024-05-03T17:52:41.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:40 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2246605358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:52:41.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:40 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2246605358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:52:41.176 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:40 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2246605358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:52:41.536 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T17:52:41.776 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:41.898 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:41 smithi122 ceph-mon[29744]: pgmap v288: 97 pgs: 97 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T17:52:42.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:41 smithi149 ceph-mon[33409]: pgmap v288: 97 pgs: 97 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T17:52:42.058 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:41 smithi192 ceph-mon[33753]: pgmap v288: 97 pgs: 97 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T17:52:42.365 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:42 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i[64511]: 2024-05-03T17:52:42.062+0000 7fbfdff19700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-03T17:52:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.499+0000 7ff8522df700 1 -- 172.21.15.122:0/2479912352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c108850 msgr2=0x7ff84c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.499+0000 7ff8522df700 1 --2- 172.21.15.122:0/2479912352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c108850 0x7ff84c1103d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff84400b0d0 tx=0x7ff84400b3e0 comp rx=0 tx=0).stop 2024-05-03T17:52:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.499+0000 7ff8522df700 1 -- 172.21.15.122:0/2479912352 shutdown_connections 2024-05-03T17:52:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.499+0000 7ff8522df700 1 --2- 172.21.15.122:0/2479912352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c108850 0x7ff84c1103d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.499+0000 7ff8522df700 1 --2- 172.21.15.122:0/2479912352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff84c103750 0x7ff84c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.499+0000 7ff8522df700 1 --2- 172.21.15.122:0/2479912352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff84c104100 0x7ff84c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.499+0000 7ff8522df700 1 -- 172.21.15.122:0/2479912352 >> 172.21.15.122:0/2479912352 conn(0x7ff84c07a600 msgr2=0x7ff84c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.499+0000 7ff8522df700 1 -- 172.21.15.122:0/2479912352 shutdown_connections 2024-05-03T17:52:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.499+0000 7ff8522df700 1 -- 172.21.15.122:0/2479912352 wait complete. 2024-05-03T17:52:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.499+0000 7ff8522df700 1 Processor -- start 2024-05-03T17:52:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.499+0000 7ff8522df700 1 -- start start 2024-05-03T17:52:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff8522df700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff84c103750 0x7ff84c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff8522df700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff84c104100 0x7ff84c19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff8522df700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c108850 0x7ff84c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff8522df700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c113620 con 0x7ff84c108850 2024-05-03T17:52:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff8522df700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c1134a0 con 0x7ff84c103750 2024-05-03T17:52:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff8522df700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c1137a0 con 0x7ff84c104100 2024-05-03T17:52:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff85087c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c108850 0x7ff84c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff85087c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c108850 0x7ff84c199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37762/0 (socket says 172.21.15.122:37762) 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff85087c700 1 -- 172.21.15.122:0/3133533833 learned_addr learned my addr 172.21.15.122:0/3133533833 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff84b7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff84c104100 0x7ff84c19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff84bfff700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff84c103750 0x7ff84c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff85087c700 1 -- 172.21.15.122:0/3133533833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff84c104100 msgr2=0x7ff84c19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff85087c700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff84c104100 0x7ff84c19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff85087c700 1 -- 172.21.15.122:0/3133533833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff84c103750 msgr2=0x7ff84c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff85087c700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff84c103750 0x7ff84c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff85087c700 1 -- 172.21.15.122:0/3133533833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c19a000 con 0x7ff84c108850 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff84bfff700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff84c103750 0x7ff84c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff85087c700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c108850 0x7ff84c199820 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff844009fd0 tx=0x7ff844007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff8497fa700 1 -- 172.21.15.122:0/3133533833 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84401f070 con 0x7ff84c108850 2024-05-03T17:52:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff8497fa700 1 -- 172.21.15.122:0/3133533833 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff844003c00 con 0x7ff84c108850 2024-05-03T17:52:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff8522df700 1 -- 172.21.15.122:0/3133533833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84c19a290 con 0x7ff84c108850 2024-05-03T17:52:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff8497fa700 1 -- 172.21.15.122:0/3133533833 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84401b7b0 con 0x7ff84c108850 2024-05-03T17:52:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.500+0000 7ff8522df700 1 -- 172.21.15.122:0/3133533833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84c1a64b0 con 0x7ff84c108850 2024-05-03T17:52:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.501+0000 7ff8522df700 1 -- 172.21.15.122:0/3133533833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84c104be0 con 0x7ff84c108850 2024-05-03T17:52:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.503+0000 7ff8497fa700 1 -- 172.21.15.122:0/3133533833 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff844022070 con 0x7ff84c108850 2024-05-03T17:52:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.504+0000 7ff8497fa700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff8340b2880 0x7ff8340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.504+0000 7ff84bfff700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff8340b2880 0x7ff8340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.504+0000 7ff8497fa700 1 -- 172.21.15.122:0/3133533833 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7ff84400f020 con 0x7ff84c108850 2024-05-03T17:52:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.504+0000 7ff84bfff700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff8340b2880 0x7ff8340b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff83c005fd0 tx=0x7ff83c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:43.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.508+0000 7ff8497fa700 1 -- 172.21.15.122:0/3133533833 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff844029020 con 0x7ff84c108850 2024-05-03T17:52:43.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.751+0000 7ff8522df700 1 -- 172.21.15.122:0/3133533833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff84c19ad00 con 0x7ff84c108850 2024-05-03T17:52:43.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.752+0000 7ff8497fa700 1 -- 172.21.15.122:0/3133533833 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22283 (secure 0 0 0) 0x7ff8440ab6a0 con 0x7ff84c108850 2024-05-03T17:52:43.748 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:52:43.748 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":77,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","created":"2024-05-03T17:44:17.415571+0000","modified":"2024-05-03T17:52:37.616764+0000","last_up_change":"2024-05-03T17:51:28.839117+0000","last_in_change":"2024-05-03T17:51:07.954070+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T17:47:40.539967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T17:52:31.646919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T17:52:34.503067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4b08628-0a29-44aa-b8b9-ed9657f12be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6803","nonce":1329364610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6805","nonce":1329364610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6809","nonce":1329364610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6807","nonce":1329364610}]},"public_addr":"172.21.15.122:6803/1329364610","cluster_addr":"172.21.15.122:6805/1329364610","heartbeat_back_addr":"172.21.15.122:6809/1329364610","heartbeat_front_addr":"172.21.15.122:6807/1329364610","state":["exists","up"]},{"osd":1,"uuid":"ba550787-ce76-4944-8ebd-97e19590bb3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6801","nonce":1402463302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6803","nonce":1402463302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6807","nonce":1402463302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6805","nonce":1402463302}]},"public_addr":"172.21.15.149:6801/1402463302","cluster_addr":"172.21.15.149:6803/1402463302","heartbeat_back_addr":"172.21.15.149:6807/1402463302","heartbeat_front_addr":"172.21.15.149:6805/1402463302","state":["exists","up"]},{"osd":2,"uuid":"35ae7c47-d710-47c5-b0ec-91fa12b5be07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6801","nonce":3757191110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6803","nonce":3757191110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6807","nonce":3757191110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6805","nonce":3757191110}]},"public_addr":"172.21.15.192:6801/3757191110","cluster_addr":"172.21.15.192:6803/3757191110","heartbeat_back_addr":"172.21.15.192:6807/3757191110","heartbeat_front_addr":"172.21.15.192:6805/3757191110","state":["exists","up"]},{"osd":3,"uuid":"9973c0c1-ae73-497a-845e-664a20666df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6811","nonce":2359143764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6813","nonce":2359143764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6817","nonce":2359143764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6815","nonce":2359143764}]},"public_addr":"172.21.15.122:6811/2359143764","cluster_addr":"172.21.15.122:6813/2359143764","heartbeat_back_addr":"172.21.15.122:6817/2359143764","heartbeat_front_addr":"172.21.15.122:6815/2359143764","state":["exists","up"]},{"osd":4,"uuid":"da71dc18-94b9-4f7d-a061-7cc7b77db282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3627037532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3627037532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3627037532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3627037532}]},"public_addr":"172.21.15.149:6809/3627037532","cluster_addr":"172.21.15.149:6811/3627037532","heartbeat_back_addr":"172.21.15.149:6815/3627037532","heartbeat_front_addr":"172.21.15.149:6813/3627037532","state":["exists","up"]},{"osd":5,"uuid":"bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6809","nonce":455779485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6811","nonce":455779485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6815","nonce":455779485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6813","nonce":455779485}]},"public_addr":"172.21.15.192:6809/455779485","cluster_addr":"172.21.15.192:6811/455779485","heartbeat_back_addr":"172.21.15.192:6815/455779485","heartbeat_front_addr":"172.21.15.192:6813/455779485","state":["exists","up"]},{"osd":6,"uuid":"4150bfe5-a0e4-4bae-914a-75598f017519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6819","nonce":3419421998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6821","nonce":3419421998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6825","nonce":3419421998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6823","nonce":3419421998}]},"public_addr":"172.21.15.122:6819/3419421998","cluster_addr":"172.21.15.122:6821/3419421998","heartbeat_back_addr":"172.21.15.122:6825/3419421998","heartbeat_front_addr":"172.21.15.122:6823/3419421998","state":["exists","up"]},{"osd":7,"uuid":"71e01812-e511-44a0-a143-2afca510ae8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6817","nonce":139747549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6819","nonce":139747549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6823","nonce":139747549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6821","nonce":139747549}]},"public_addr":"172.21.15.149:6817/139747549","cluster_addr":"172.21.15.149:6819/139747549","heartbeat_back_addr":"172.21.15.149:6823/139747549","heartbeat_front_addr":"172.21.15.149:6821/139747549","state":["exists","up"]},{"osd":8,"uuid":"5294516d-fac6-41d0-a635-6bc1d7eca585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6817","nonce":2080816744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6819","nonce":2080816744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6823","nonce":2080816744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6821","nonce":2080816744}]},"public_addr":"172.21.15.192:6817/2080816744","cluster_addr":"172.21.15.192:6819/2080816744","heartbeat_back_addr":"172.21.15.192:6823/2080816744","heartbeat_front_addr":"172.21.15.192:6821/2080816744","state":["exists","up"]},{"osd":9,"uuid":"f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6827","nonce":1773691421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6829","nonce":1773691421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6833","nonce":1773691421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6831","nonce":1773691421}]},"public_addr":"172.21.15.122:6827/1773691421","cluster_addr":"172.21.15.122:6829/1773691421","heartbeat_back_addr":"172.21.15.122:6833/1773691421","heartbeat_front_addr":"172.21.15.122:6831/1773691421","state":["exists","up"]},{"osd":10,"uuid":"af88e50f-98f1-460d-856b-a920dd2ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6825","nonce":1954186137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6827","nonce":1954186137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6831","nonce":1954186137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6829","nonce":1954186137}]},"public_addr":"172.21.15.149:6825/1954186137","cluster_addr":"172.21.15.149:6827/1954186137","heartbeat_back_addr":"172.21.15.149:6831/1954186137","heartbeat_front_addr":"172.21.15.149:6829/1954186137","state":["exists","up"]},{"osd":11,"uuid":"b54c1b4d-cdb5-42d9-979a-47f9a870b5cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6825","nonce":2098474209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6827","nonce":2098474209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6831","nonce":2098474209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6829","nonce":2098474209}]},"public_addr":"172.21.15.192:6825/2098474209","cluster_addr":"172.21.15.192:6827/2098474209","heartbeat_back_addr":"172.21.15.192:6831/2098474209","heartbeat_front_addr":"172.21.15.192:6829/2098474209","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:46:50.049769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:13.518485+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:37.473023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:01.809640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:26.648514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:51.268056+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:16.294834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:42.079065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:07.609271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:33.885127+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:59.982708+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:51:26.461369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/2089907395":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6801/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:0/2001860625":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/1888284988":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/256574472":"2024-05-04T17:44:52.244049+0000","172.21.15.122:6801/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/64546794":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/584846864":"2024-05-04T17:45:18.454406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T17:52:43.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 -- 172.21.15.122:0/3133533833 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff8340b2880 msgr2=0x7ff8340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:43.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff8340b2880 0x7ff8340b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff83c005fd0 tx=0x7ff83c005e30 comp rx=0 tx=0).stop 2024-05-03T17:52:43.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 -- 172.21.15.122:0/3133533833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c108850 msgr2=0x7ff84c199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:43.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c108850 0x7ff84c199820 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff844009fd0 tx=0x7ff844007a20 comp rx=0 tx=0).stop 2024-05-03T17:52:43.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 -- 172.21.15.122:0/3133533833 shutdown_connections 2024-05-03T17:52:43.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff8340b2880 0x7ff8340b4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:43.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c108850 0x7ff84c199820 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:43.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff84c103750 0x7ff84c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:43.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 --2- 172.21.15.122:0/3133533833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff84c104100 0x7ff84c19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:43.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 -- 172.21.15.122:0/3133533833 >> 172.21.15.122:0/3133533833 conn(0x7ff84c07a600 msgr2=0x7ff84c1014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:43.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 -- 172.21.15.122:0/3133533833 shutdown_connections 2024-05-03T17:52:43.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:43.755+0000 7ff8522df700 1 -- 172.21.15.122:0/3133533833 wait complete. 2024-05-03T17:52:43.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:43 smithi122 ceph-mon[29744]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-03T17:52:44.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:43 smithi149 ceph-mon[33409]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-03T17:52:44.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:43 smithi192 ceph-mon[33753]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-03T17:52:44.527 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-03T17:52:44.772 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:44.805 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:44 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3133533833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:52:45.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:44 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3133533833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:52:45.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:44 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3133533833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:52:46.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:45 smithi149 ceph-mon[33409]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T17:52:46.063 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:45 smithi122 ceph-mon[29744]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T17:52:46.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:45 smithi192 ceph-mon[33753]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T17:52:46.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.461+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2805701224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb0c109510 msgr2=0x7ffb0c1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:46.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.461+0000 7ffb12a3f700 1 --2- 172.21.15.122:0/2805701224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb0c109510 0x7ffb0c1098f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009990 tx=0x7ffafc009ca0 comp rx=0 tx=0).stop 2024-05-03T17:52:46.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.461+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2805701224 shutdown_connections 2024-05-03T17:52:46.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.461+0000 7ffb12a3f700 1 --2- 172.21.15.122:0/2805701224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c06c5b0 0x7ffb0c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:46.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.461+0000 7ffb12a3f700 1 --2- 172.21.15.122:0/2805701224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb0c109510 0x7ffb0c1098f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:46.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.461+0000 7ffb12a3f700 1 --2- 172.21.15.122:0/2805701224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c06bbf0 0x7ffb0c06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:46.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.461+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2805701224 >> 172.21.15.122:0/2805701224 conn(0x7ffb0c079370 msgr2=0x7ffb0c079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:46.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.461+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2805701224 shutdown_connections 2024-05-03T17:52:46.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.461+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2805701224 wait complete. 2024-05-03T17:52:46.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb12a3f700 1 Processor -- start 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb12a3f700 1 -- start start 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb12a3f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c06bbf0 0x7ffb0c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb12a3f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c06c5b0 0x7ffb0c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb12a3f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb0c109510 0x7ffb0c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb12a3f700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c113620 con 0x7ffb0c109510 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb12a3f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c1134a0 con 0x7ffb0c06bbf0 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb12a3f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c1137a0 con 0x7ffb0c06c5b0 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb10fdc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb0c109510 0x7ffb0c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb10fdc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb0c109510 0x7ffb0c199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37780/0 (socket says 172.21.15.122:37780) 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb10fdc700 1 -- 172.21.15.122:0/2167754317 learned_addr learned my addr 172.21.15.122:0/2167754317 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb0b7fe700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c06c5b0 0x7ffb0c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb0bfff700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c06bbf0 0x7ffb0c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:46.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb10fdc700 1 -- 172.21.15.122:0/2167754317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c06c5b0 msgr2=0x7ffb0c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb10fdc700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c06c5b0 0x7ffb0c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb10fdc700 1 -- 172.21.15.122:0/2167754317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c06bbf0 msgr2=0x7ffb0c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb10fdc700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c06bbf0 0x7ffb0c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb10fdc700 1 -- 172.21.15.122:0/2167754317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0c19a020 con 0x7ffb0c109510 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.462+0000 7ffb10fdc700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb0c109510 0x7ffb0c199840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb04007d70 tx=0x7ffb0400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.463+0000 7ffb097fa700 1 -- 172.21.15.122:0/2167754317 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04016020 con 0x7ffb0c109510 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.463+0000 7ffb097fa700 1 -- 172.21.15.122:0/2167754317 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0400ad40 con 0x7ffb0c109510 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.463+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2167754317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb0c19a310 con 0x7ffb0c109510 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.463+0000 7ffb097fa700 1 -- 172.21.15.122:0/2167754317 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb040115e0 con 0x7ffb0c109510 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.463+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2167754317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb0c1a6520 con 0x7ffb0c109510 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.464+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2167754317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb0c10b9f0 con 0x7ffb0c109510 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.465+0000 7ffb097fa700 1 -- 172.21.15.122:0/2167754317 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ffb04019020 con 0x7ffb0c109510 2024-05-03T17:52:46.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.465+0000 7ffb097fa700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaf40b2880 0x7ffaf40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:46.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.466+0000 7ffb0bfff700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaf40b2880 0x7ffaf40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:46.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.466+0000 7ffb097fa700 1 -- 172.21.15.122:0/2167754317 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7ffb040e2480 con 0x7ffb0c109510 2024-05-03T17:52:46.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.466+0000 7ffb0bfff700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaf40b2880 0x7ffaf40b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffafc006010 tx=0x7ffafc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:46.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.469+0000 7ffb097fa700 1 -- 172.21.15.122:0/2167754317 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ffb04020020 con 0x7ffb0c109510 2024-05-03T17:52:46.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.721+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2167754317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ffb0c104bd0 con 0x7ffb0c109510 2024-05-03T17:52:46.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.749+0000 7ffb097fa700 1 -- 172.21.15.122:0/2167754317 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7ffb040aa2f0 con 0x7ffb0c109510 2024-05-03T17:52:46.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2167754317 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaf40b2880 msgr2=0x7ffaf40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:46.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaf40b2880 0x7ffaf40b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffafc006010 tx=0x7ffafc01a040 comp rx=0 tx=0).stop 2024-05-03T17:52:46.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2167754317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb0c109510 msgr2=0x7ffb0c199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:46.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb0c109510 0x7ffb0c199840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb04007d70 tx=0x7ffb0400a340 comp rx=0 tx=0).stop 2024-05-03T17:52:46.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2167754317 shutdown_connections 2024-05-03T17:52:46.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaf40b2880 0x7ffaf40b4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:46.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb0c109510 0x7ffb0c199840 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:46.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c06bbf0 0x7ffb0c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:46.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 --2- 172.21.15.122:0/2167754317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c06c5b0 0x7ffb0c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:46.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2167754317 >> 172.21.15.122:0/2167754317 conn(0x7ffb0c079370 msgr2=0x7ffb0c100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:46.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2167754317 shutdown_connections 2024-05-03T17:52:46.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:46.752+0000 7ffb12a3f700 1 -- 172.21.15.122:0/2167754317 wait complete. 2024-05-03T17:52:47.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:46 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2167754317' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T17:52:47.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:46 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2167754317' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T17:52:47.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:46 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2167754317' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T17:52:47.393 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-03T17:52:47.394 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-03T17:52:47.633 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:48.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:47 smithi122 ceph-mon[29744]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-03T17:52:48.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:47 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2167754317' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T17:52:48.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:47 smithi122 ceph-mon[29744]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:48.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:47 smithi149 ceph-mon[33409]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-03T17:52:48.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:47 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2167754317' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T17:52:48.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:47 smithi149 ceph-mon[33409]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:48.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:47 smithi192 ceph-mon[33753]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-03T17:52:48.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:47 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2167754317' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T17:52:48.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:47 smithi192 ceph-mon[33753]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:49.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.396+0000 7ff744240700 1 -- 172.21.15.122:0/3445013483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff73c1076c0 msgr2=0x7ff73c10f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.396+0000 7ff744240700 1 --2- 172.21.15.122:0/3445013483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff73c1076c0 0x7ff73c10f290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff73400b0d0 tx=0x7ff73400b3e0 comp rx=0 tx=0).stop 2024-05-03T17:52:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.396+0000 7ff744240700 1 -- 172.21.15.122:0/3445013483 shutdown_connections 2024-05-03T17:52:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.396+0000 7ff744240700 1 --2- 172.21.15.122:0/3445013483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff73c1076c0 0x7ff73c10f290 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.396+0000 7ff744240700 1 --2- 172.21.15.122:0/3445013483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73c102630 0x7ff73c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.396+0000 7ff744240700 1 --2- 172.21.15.122:0/3445013483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff73c102fe0 0x7ff73c106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.396+0000 7ff744240700 1 -- 172.21.15.122:0/3445013483 >> 172.21.15.122:0/3445013483 conn(0x7ff73c079370 msgr2=0x7ff73c079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.396+0000 7ff744240700 1 -- 172.21.15.122:0/3445013483 shutdown_connections 2024-05-03T17:52:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.396+0000 7ff744240700 1 -- 172.21.15.122:0/3445013483 wait complete. 2024-05-03T17:52:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff744240700 1 Processor -- start 2024-05-03T17:52:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff744240700 1 -- start start 2024-05-03T17:52:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff744240700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73c102630 0x7ff73c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff744240700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff73c102fe0 0x7ff73c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff744240700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff73c1076c0 0x7ff73c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff744240700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c1125b0 con 0x7ff73c1076c0 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff744240700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c112430 con 0x7ff73c102630 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff744240700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c112730 con 0x7ff73c102fe0 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff7427dd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff73c1076c0 0x7ff73c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff7427dd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff73c1076c0 0x7ff73c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52672/0 (socket says 172.21.15.122:52672) 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff7427dd700 1 -- 172.21.15.122:0/3530408037 learned_addr learned my addr 172.21.15.122:0/3530408037 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff741fdc700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73c102630 0x7ff73c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff7417db700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff73c102fe0 0x7ff73c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff7427dd700 1 -- 172.21.15.122:0/3530408037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff73c102fe0 msgr2=0x7ff73c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff7427dd700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff73c102fe0 0x7ff73c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff7427dd700 1 -- 172.21.15.122:0/3530408037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73c102630 msgr2=0x7ff73c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:49.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff7427dd700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73c102630 0x7ff73c19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:49.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff7427dd700 1 -- 172.21.15.122:0/3530408037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c199050 con 0x7ff73c1076c0 2024-05-03T17:52:49.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.397+0000 7ff7417db700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff73c102fe0 0x7ff73c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:52:49.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.398+0000 7ff7427dd700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff73c1076c0 0x7ff73c1988f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7340079a0 tx=0x7ff7340079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:49.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.398+0000 7ff732ffd700 1 -- 172.21.15.122:0/3530408037 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73401f070 con 0x7ff73c1076c0 2024-05-03T17:52:49.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.398+0000 7ff732ffd700 1 -- 172.21.15.122:0/3530408037 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff734003c00 con 0x7ff73c1076c0 2024-05-03T17:52:49.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.398+0000 7ff744240700 1 -- 172.21.15.122:0/3530408037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73c1992e0 con 0x7ff73c1076c0 2024-05-03T17:52:49.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.398+0000 7ff732ffd700 1 -- 172.21.15.122:0/3530408037 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73401b6a0 con 0x7ff73c1076c0 2024-05-03T17:52:49.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.398+0000 7ff744240700 1 -- 172.21.15.122:0/3530408037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff73c1a5600 con 0x7ff73c1076c0 2024-05-03T17:52:49.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.399+0000 7ff744240700 1 -- 172.21.15.122:0/3530408037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff73c103ac0 con 0x7ff73c1076c0 2024-05-03T17:52:49.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.400+0000 7ff732ffd700 1 -- 172.21.15.122:0/3530408037 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff734022070 con 0x7ff73c1076c0 2024-05-03T17:52:49.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.401+0000 7ff732ffd700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff7280b2880 0x7ff7280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:49.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.401+0000 7ff741fdc700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff7280b2880 0x7ff7280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:49.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.401+0000 7ff732ffd700 1 -- 172.21.15.122:0/3530408037 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7ff73400f020 con 0x7ff73c1076c0 2024-05-03T17:52:49.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.401+0000 7ff741fdc700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff7280b2880 0x7ff7280b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff738005950 tx=0x7ff73800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:49.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.404+0000 7ff732ffd700 1 -- 172.21.15.122:0/3530408037 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff734029020 con 0x7ff73c1076c0 2024-05-03T17:52:49.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.661+0000 7ff744240700 1 -- 172.21.15.122:0/3530408037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ff73c10a3c0 con 0x7ff73c1076c0 2024-05-03T17:52:49.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.762+0000 7ff732ffd700 1 -- 172.21.15.122:0/3530408037 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7ff7340ab640 con 0x7ff73c1076c0 2024-05-03T17:52:50.728 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:49 smithi192 ceph-mon[33753]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-03T17:52:50.728 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:49 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3530408037' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-03T17:52:50.729 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:49 smithi149 ceph-mon[33409]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-03T17:52:50.729 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:49 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3530408037' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-03T17:52:50.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 -- 172.21.15.122:0/3530408037 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff7280b2880 msgr2=0x7ff7280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:50.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff7280b2880 0x7ff7280b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff738005950 tx=0x7ff73800a300 comp rx=0 tx=0).stop 2024-05-03T17:52:50.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 -- 172.21.15.122:0/3530408037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff73c1076c0 msgr2=0x7ff73c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:50.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff73c1076c0 0x7ff73c1988f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7340079a0 tx=0x7ff7340079d0 comp rx=0 tx=0).stop 2024-05-03T17:52:50.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 -- 172.21.15.122:0/3530408037 shutdown_connections 2024-05-03T17:52:50.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff7280b2880 0x7ff7280b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:50.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff73c1076c0 0x7ff73c1988f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:50.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff73c102630 0x7ff73c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:50.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 --2- 172.21.15.122:0/3530408037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff73c102fe0 0x7ff73c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:50.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 -- 172.21.15.122:0/3530408037 >> 172.21.15.122:0/3530408037 conn(0x7ff73c079370 msgr2=0x7ff73c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:50.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 -- 172.21.15.122:0/3530408037 shutdown_connections 2024-05-03T17:52:50.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:49.764+0000 7ff744240700 1 -- 172.21.15.122:0/3530408037 wait complete. 2024-05-03T17:52:50.731 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:49 smithi122 ceph-mon[29744]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-03T17:52:50.731 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:49 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3530408037' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-03T17:52:50.732 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-03T17:52:50.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:50 smithi122 ceph-mon[29744]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T17:52:50.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:50 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3530408037' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-03T17:52:50.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:50 smithi122 ceph-mon[29744]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T17:52:50.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:50 smithi122 ceph-mon[29744]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:50.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:50 smithi122 ceph-mon[29744]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-03T17:52:50.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:50 smithi122 ceph-mon[29744]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-03T17:52:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:50 smithi149 ceph-mon[33409]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T17:52:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:50 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3530408037' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-03T17:52:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:50 smithi149 ceph-mon[33409]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T17:52:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:50 smithi149 ceph-mon[33409]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:50 smithi149 ceph-mon[33409]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-03T17:52:51.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:50 smithi149 ceph-mon[33409]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-03T17:52:51.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:50 smithi192 ceph-mon[33753]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T17:52:51.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:50 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3530408037' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-03T17:52:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:50 smithi192 ceph-mon[33753]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T17:52:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:50 smithi192 ceph-mon[33753]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T17:52:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:50 smithi192 ceph-mon[33753]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-03T17:52:51.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:50 smithi192 ceph-mon[33753]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-03T17:52:52.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:51 smithi149 ceph-mon[33409]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-03T17:52:52.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:51 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] up:active 2024-05-03T17:52:52.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:51 smithi149 ceph-mon[33409]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T17:52:52.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:51 smithi149 ceph-mon[33409]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T17:52:52.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:51 smithi149 ceph-mon[33409]: Cluster is now healthy 2024-05-03T17:52:52.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:51 smithi149 ceph-mon[33409]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-03T17:52:52.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:51 smithi149 ceph-mon[33409]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-05-03T17:52:52.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:51 smithi149 ceph-mon[33409]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-03T17:52:52.058 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:51 smithi192 ceph-mon[33753]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-03T17:52:52.058 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:51 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] up:active 2024-05-03T17:52:52.058 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:51 smithi192 ceph-mon[33753]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T17:52:52.058 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:51 smithi192 ceph-mon[33753]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T17:52:52.058 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:51 smithi192 ceph-mon[33753]: Cluster is now healthy 2024-05-03T17:52:52.058 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:51 smithi192 ceph-mon[33753]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-03T17:52:52.058 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:51 smithi192 ceph-mon[33753]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-05-03T17:52:52.058 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:51 smithi192 ceph-mon[33753]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-03T17:52:52.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:51 smithi122 ceph-mon[29744]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-03T17:52:52.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:51 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] up:active 2024-05-03T17:52:52.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:51 smithi122 ceph-mon[29744]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T17:52:52.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:51 smithi122 ceph-mon[29744]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T17:52:52.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:51 smithi122 ceph-mon[29744]: Cluster is now healthy 2024-05-03T17:52:52.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:51 smithi122 ceph-mon[29744]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-03T17:52:52.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:51 smithi122 ceph-mon[29744]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-05-03T17:52:52.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:51 smithi122 ceph-mon[29744]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-03T17:52:52.365 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:52 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i[64511]: 2024-05-03T17:52:52.062+0000 7fbfdff19700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-03T17:52:52.365 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 17:52:52 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i[64511]: 2024-05-03T17:52:52.062+0000 7fbfdff19700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-03T17:52:52.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.756+0000 7fe8bced3700 1 -- 172.21.15.122:0/2896459493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8b8109520 msgr2=0x7fe8b8109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:52.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.756+0000 7fe8bced3700 1 --2- 172.21.15.122:0/2896459493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8b8109520 0x7fe8b8109900 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009990 tx=0x7fe8a8009ca0 comp rx=0 tx=0).stop 2024-05-03T17:52:52.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.756+0000 7fe8bced3700 1 -- 172.21.15.122:0/2896459493 shutdown_connections 2024-05-03T17:52:52.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.756+0000 7fe8bced3700 1 --2- 172.21.15.122:0/2896459493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8b8102450 0x7fe8b8110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:52.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.756+0000 7fe8bced3700 1 --2- 172.21.15.122:0/2896459493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8b8109520 0x7fe8b8109900 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:52.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.756+0000 7fe8bced3700 1 --2- 172.21.15.122:0/2896459493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8b8101a90 0x7fe8b8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:52.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.756+0000 7fe8bced3700 1 -- 172.21.15.122:0/2896459493 >> 172.21.15.122:0/2896459493 conn(0x7fe8b8078a10 msgr2=0x7fe8b80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:52.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.756+0000 7fe8bced3700 1 -- 172.21.15.122:0/2896459493 shutdown_connections 2024-05-03T17:52:52.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.756+0000 7fe8bced3700 1 -- 172.21.15.122:0/2896459493 wait complete. 2024-05-03T17:52:52.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8bced3700 1 Processor -- start 2024-05-03T17:52:52.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8bced3700 1 -- start start 2024-05-03T17:52:52.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8bced3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8b8101a90 0x7fe8b819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:52.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8bced3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8b8102450 0x7fe8b819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:52.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8bced3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8b8109520 0x7fe8b81999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:52.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8bced3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81136d0 con 0x7fe8b8101a90 2024-05-03T17:52:52.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8bced3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8113550 con 0x7fe8b8109520 2024-05-03T17:52:52.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8bced3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8113850 con 0x7fe8b8102450 2024-05-03T17:52:52.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8b659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8b8101a90 0x7fe8b819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:52.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8add9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8b8102450 0x7fe8b819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:52.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8b659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8b8101a90 0x7fe8b819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52682/0 (socket says 172.21.15.122:52682) 2024-05-03T17:52:52.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8b659c700 1 -- 172.21.15.122:0/2408336436 learned_addr learned my addr 172.21.15.122:0/2408336436 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:52.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8b6d9d700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8b8109520 0x7fe8b81999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:52.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8b659c700 1 -- 172.21.15.122:0/2408336436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8b8102450 msgr2=0x7fe8b819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:52.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8b659c700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8b8102450 0x7fe8b819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:52.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8b659c700 1 -- 172.21.15.122:0/2408336436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8b8109520 msgr2=0x7fe8b81999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:52.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8b659c700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8b8109520 0x7fe8b81999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:52.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8b659c700 1 -- 172.21.15.122:0/2408336436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b819a110 con 0x7fe8b8101a90 2024-05-03T17:52:52.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.757+0000 7fe8add9b700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8b8102450 0x7fe8b819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T17:52:52.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.758+0000 7fe8b659c700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8b8101a90 0x7fe8b819f7f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000c00 tx=0x7fe8a8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:52.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.758+0000 7fe8affff700 1 -- 172.21.15.122:0/2408336436 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8027070 con 0x7fe8b8101a90 2024-05-03T17:52:52.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.758+0000 7fe8affff700 1 -- 172.21.15.122:0/2408336436 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8a8022470 con 0x7fe8b8101a90 2024-05-03T17:52:52.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.758+0000 7fe8bced3700 1 -- 172.21.15.122:0/2408336436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b819a3a0 con 0x7fe8b8101a90 2024-05-03T17:52:52.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.758+0000 7fe8affff700 1 -- 172.21.15.122:0/2408336436 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8017650 con 0x7fe8b8101a90 2024-05-03T17:52:52.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.758+0000 7fe8bced3700 1 -- 172.21.15.122:0/2408336436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b81a66c0 con 0x7fe8b8101a90 2024-05-03T17:52:52.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.759+0000 7fe8bced3700 1 -- 172.21.15.122:0/2408336436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b810ba00 con 0x7fe8b8101a90 2024-05-03T17:52:52.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.760+0000 7fe8affff700 1 -- 172.21.15.122:0/2408336436 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fe8a802a070 con 0x7fe8b8101a90 2024-05-03T17:52:52.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.761+0000 7fe8affff700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe8a40b2880 0x7fe8a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:52.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.761+0000 7fe8add9b700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe8a40b2880 0x7fe8a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:52.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.761+0000 7fe8affff700 1 -- 172.21.15.122:0/2408336436 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fe8a80e30a0 con 0x7fe8b8101a90 2024-05-03T17:52:52.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.761+0000 7fe8add9b700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe8a40b2880 0x7fe8a40b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8108c60 tx=0x7fe8a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:52.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:52.764+0000 7fe8affff700 1 -- 172.21.15.122:0/2408336436 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe8a8013070 con 0x7fe8b8101a90 2024-05-03T17:52:52.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:52 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] up:active 2024-05-03T17:52:52.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:52 smithi122 ceph-mon[29744]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-03T17:52:53.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.021+0000 7fe8bced3700 1 -- 172.21.15.122:0/2408336436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe8b8104db0 con 0x7fe8b8101a90 2024-05-03T17:52:53.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:52 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] up:active 2024-05-03T17:52:53.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:52 smithi149 ceph-mon[33409]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-03T17:52:53.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:52 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] up:active 2024-05-03T17:52:53.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:52 smithi192 ceph-mon[33753]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-03T17:52:53.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.758+0000 7fe8affff700 1 -- 172.21.15.122:0/2408336436 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fe8a80aafa0 con 0x7fe8b8101a90 2024-05-03T17:52:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 -- 172.21.15.122:0/2408336436 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe8a40b2880 msgr2=0x7fe8a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe8a40b2880 0x7fe8a40b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8108c60 tx=0x7fe8a0009260 comp rx=0 tx=0).stop 2024-05-03T17:52:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 -- 172.21.15.122:0/2408336436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8b8101a90 msgr2=0x7fe8b819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8b8101a90 0x7fe8b819f7f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000c00 tx=0x7fe8a8009580 comp rx=0 tx=0).stop 2024-05-03T17:52:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 -- 172.21.15.122:0/2408336436 shutdown_connections 2024-05-03T17:52:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe8a40b2880 0x7fe8a40b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8b8109520 0x7fe8b81999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:53.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8b8101a90 0x7fe8b819f7f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:53.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 --2- 172.21.15.122:0/2408336436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8b8102450 0x7fe8b819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:53.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 -- 172.21.15.122:0/2408336436 >> 172.21.15.122:0/2408336436 conn(0x7fe8b8078a10 msgr2=0x7fe8b8100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:53.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 -- 172.21.15.122:0/2408336436 shutdown_connections 2024-05-03T17:52:53.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:53.762+0000 7fe8bced3700 1 -- 172.21.15.122:0/2408336436 wait complete. 2024-05-03T17:52:54.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:53 smithi149 ceph-mon[33409]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 2 op/s 2024-05-03T17:52:54.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:53 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2408336436' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T17:52:54.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:53 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2408336436' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T17:52:54.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:53 smithi149 ceph-mon[33409]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T17:52:54.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:53 smithi122 ceph-mon[29744]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 2 op/s 2024-05-03T17:52:54.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:53 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2408336436' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T17:52:54.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:53 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2408336436' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T17:52:54.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:53 smithi122 ceph-mon[29744]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T17:52:54.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:53 smithi192 ceph-mon[33753]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 2 op/s 2024-05-03T17:52:54.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:53 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2408336436' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T17:52:54.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:53 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2408336436' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T17:52:54.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:53 smithi192 ceph-mon[33753]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T17:52:54.546 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-03T17:52:54.788 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:55.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:55 smithi122 ceph-mon[29744]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-03T17:52:56.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:55 smithi149 ceph-mon[33409]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-03T17:52:56.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:55 smithi192 ceph-mon[33753]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-03T17:52:56.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.593+0000 7f94b1a4f700 1 -- 172.21.15.122:0/621286687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94ac109520 msgr2=0x7f94ac109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:56.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.593+0000 7f94b1a4f700 1 --2- 172.21.15.122:0/621286687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94ac109520 0x7f94ac109900 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f949c00b0a0 tx=0x7f949c00b3b0 comp rx=0 tx=0).stop 2024-05-03T17:52:56.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.593+0000 7f94b1a4f700 1 -- 172.21.15.122:0/621286687 shutdown_connections 2024-05-03T17:52:56.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.593+0000 7f94b1a4f700 1 --2- 172.21.15.122:0/621286687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ac102450 0x7f94ac110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:56.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.593+0000 7f94b1a4f700 1 --2- 172.21.15.122:0/621286687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94ac109520 0x7f94ac109900 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:56.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.593+0000 7f94b1a4f700 1 --2- 172.21.15.122:0/621286687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94ac101a90 0x7f94ac101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:56.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.593+0000 7f94b1a4f700 1 -- 172.21.15.122:0/621286687 >> 172.21.15.122:0/621286687 conn(0x7f94ac078a10 msgr2=0x7f94ac0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:56.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.593+0000 7f94b1a4f700 1 -- 172.21.15.122:0/621286687 shutdown_connections 2024-05-03T17:52:56.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.593+0000 7f94b1a4f700 1 -- 172.21.15.122:0/621286687 wait complete. 2024-05-03T17:52:56.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94b1a4f700 1 Processor -- start 2024-05-03T17:52:56.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94b1a4f700 1 -- start start 2024-05-03T17:52:56.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94b1a4f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ac101a90 0x7f94ac19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:56.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94b1a4f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94ac102450 0x7f94ac19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:56.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94b1a4f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94ac109520 0x7f94ac1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:56.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94b1a4f700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac1136b0 con 0x7f94ac109520 2024-05-03T17:52:56.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94b1a4f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac113530 con 0x7f94ac101a90 2024-05-03T17:52:56.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94b1a4f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac113830 con 0x7f94ac102450 2024-05-03T17:52:56.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94ab7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94ac109520 0x7f94ac1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:56.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94ab7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94ac109520 0x7f94ac1999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52690/0 (socket says 172.21.15.122:52690) 2024-05-03T17:52:56.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94ab7fe700 1 -- 172.21.15.122:0/3039135493 learned_addr learned my addr 172.21.15.122:0/3039135493 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:52:56.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94aaffd700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ac101a90 0x7f94ac19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94aa7fc700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94ac102450 0x7f94ac19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94ab7fe700 1 -- 172.21.15.122:0/3039135493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94ac102450 msgr2=0x7f94ac19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94ab7fe700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94ac102450 0x7f94ac19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94ab7fe700 1 -- 172.21.15.122:0/3039135493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ac101a90 msgr2=0x7f94ac19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94ab7fe700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ac101a90 0x7f94ac19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.594+0000 7f94ab7fe700 1 -- 172.21.15.122:0/3039135493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac19a150 con 0x7f94ac109520 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.595+0000 7f94ab7fe700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94ac109520 0x7f94ac1999f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f94a400c900 tx=0x7f94a400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.595+0000 7f94b0a4d700 1 -- 172.21.15.122:0/3039135493 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4007b00 con 0x7f94ac109520 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.595+0000 7f94b0a4d700 1 -- 172.21.15.122:0/3039135493 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a4007ca0 con 0x7f94ac109520 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.595+0000 7f94b1a4f700 1 -- 172.21.15.122:0/3039135493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ac19a440 con 0x7f94ac109520 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.595+0000 7f94b0a4d700 1 -- 172.21.15.122:0/3039135493 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4018660 con 0x7f94ac109520 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.595+0000 7f94b1a4f700 1 -- 172.21.15.122:0/3039135493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ac1a6700 con 0x7f94ac109520 2024-05-03T17:52:56.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.596+0000 7f94b1a4f700 1 -- 172.21.15.122:0/3039135493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac10ba00 con 0x7f94ac109520 2024-05-03T17:52:56.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.597+0000 7f94b0a4d700 1 -- 172.21.15.122:0/3039135493 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f94a400f450 con 0x7f94ac109520 2024-05-03T17:52:56.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.598+0000 7f94b0a4d700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f94980b2900 0x7f94980b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:52:56.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.598+0000 7f94aaffd700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f94980b2900 0x7f94980b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:52:56.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.598+0000 7f94b0a4d700 1 -- 172.21.15.122:0/3039135493 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f94a40e2070 con 0x7f94ac109520 2024-05-03T17:52:56.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.598+0000 7f94aaffd700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f94980b2900 0x7f94980b4dc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f949c00b830 tx=0x7f949c006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:52:56.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.602+0000 7f94b0a4d700 1 -- 172.21.15.122:0/3039135493 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f94a4014030 con 0x7f94ac109520 2024-05-03T17:52:56.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:56.865+0000 7f94b1a4f700 1 -- 172.21.15.122:0/3039135493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f94ac19af20 con 0x7f94ac109520 2024-05-03T17:52:57.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.799+0000 7f94b0a4d700 1 -- 172.21.15.122:0/3039135493 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f94a40a9ee0 con 0x7f94ac109520 2024-05-03T17:52:57.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 -- 172.21.15.122:0/3039135493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f94980b2900 msgr2=0x7f94980b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:57.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f94980b2900 0x7f94980b4dc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f949c00b830 tx=0x7f949c006210 comp rx=0 tx=0).stop 2024-05-03T17:52:57.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 -- 172.21.15.122:0/3039135493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94ac109520 msgr2=0x7f94ac1999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:52:57.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94ac109520 0x7f94ac1999f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f94a400c900 tx=0x7f94a400ccc0 comp rx=0 tx=0).stop 2024-05-03T17:52:57.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 -- 172.21.15.122:0/3039135493 shutdown_connections 2024-05-03T17:52:57.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f94980b2900 0x7f94980b4dc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:57.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94ac109520 0x7f94ac1999f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:57.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ac101a90 0x7f94ac19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:57.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 --2- 172.21.15.122:0/3039135493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94ac102450 0x7f94ac19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:52:57.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 -- 172.21.15.122:0/3039135493 >> 172.21.15.122:0/3039135493 conn(0x7f94ac078a10 msgr2=0x7f94ac100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:52:57.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 -- 172.21.15.122:0/3039135493 shutdown_connections 2024-05-03T17:52:57.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:52:57.802+0000 7f94b1a4f700 1 -- 172.21.15.122:0/3039135493 wait complete. 2024-05-03T17:52:58.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:57 smithi192 ceph-mon[33753]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-03T17:52:58.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:57 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3039135493' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T17:52:58.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:57 smithi122 ceph-mon[29744]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-03T17:52:58.130 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:57 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3039135493' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T17:52:58.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:57 smithi149 ceph-mon[33409]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-03T17:52:58.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:57 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3039135493' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T17:52:58.558 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-03T17:52:58.558 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-03T17:52:58.817 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:52:59.051 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:58 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3039135493' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T17:52:59.051 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:58 smithi122 ceph-mon[29744]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T17:52:59.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:58 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3039135493' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T17:52:59.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:58 smithi192 ceph-mon[33753]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T17:52:59.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:58 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3039135493' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T17:52:59.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:58 smithi149 ceph-mon[33409]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T17:53:00.055 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:52:59 smithi122 ceph-mon[29744]: pgmap v297: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-03T17:53:00.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:52:59 smithi192 ceph-mon[33753]: pgmap v297: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-03T17:53:00.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:52:59 smithi149 ceph-mon[33409]: pgmap v297: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-03T17:53:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.639+0000 7f6983f64700 1 -- 172.21.15.122:0/363981301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c102b00 msgr2=0x7f697c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.639+0000 7f6983f64700 1 --2- 172.21.15.122:0/363981301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c102b00 0x7f697c102f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f696c009990 tx=0x7f696c009ca0 comp rx=0 tx=0).stop 2024-05-03T17:53:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.639+0000 7f6983f64700 1 -- 172.21.15.122:0/363981301 shutdown_connections 2024-05-03T17:53:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.639+0000 7f6983f64700 1 --2- 172.21.15.122:0/363981301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f697c1034c0 0x7f697c10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.639+0000 7f6983f64700 1 --2- 172.21.15.122:0/363981301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c108670 0x7f697c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.639+0000 7f6983f64700 1 --2- 172.21.15.122:0/363981301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c102b00 0x7f697c102f80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.639+0000 7f6983f64700 1 -- 172.21.15.122:0/363981301 >> 172.21.15.122:0/363981301 conn(0x7f697c0789a0 msgr2=0x7f697c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.639+0000 7f6983f64700 1 -- 172.21.15.122:0/363981301 shutdown_connections 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.639+0000 7f6983f64700 1 -- 172.21.15.122:0/363981301 wait complete. 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.639+0000 7f6983f64700 1 Processor -- start 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6983f64700 1 -- start start 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6983f64700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c102b00 0x7f697c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6983f64700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f697c1034c0 0x7f697c199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6983f64700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c108670 0x7f697c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6983f64700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f697c112870 con 0x7f697c108670 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6983f64700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1126f0 con 0x7f697c1034c0 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6983f64700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1129f0 con 0x7f697c102b00 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6982501700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c108670 0x7f697c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:00.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6982501700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c108670 0x7f697c1995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53834/0 (socket says 172.21.15.122:53834) 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6982501700 1 -- 172.21.15.122:0/890399275 learned_addr learned my addr 172.21.15.122:0/890399275 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6981d00700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c102b00 0x7f697c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f69814ff700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f697c1034c0 0x7f697c199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6982501700 1 -- 172.21.15.122:0/890399275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c102b00 msgr2=0x7f697c198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6982501700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c102b00 0x7f697c198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6982501700 1 -- 172.21.15.122:0/890399275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f697c1034c0 msgr2=0x7f697c199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6982501700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f697c1034c0 0x7f697c199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6982501700 1 -- 172.21.15.122:0/890399275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c1a8390 con 0x7f697c108670 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f69814ff700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f697c1034c0 0x7f697c199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6982501700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c108670 0x7f697c1995a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f697400c970 tx=0x7f697400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.640+0000 7f6972ffd700 1 -- 172.21.15.122:0/890399275 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974004a30 con 0x7f697c108670 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.641+0000 7f6983f64700 1 -- 172.21.15.122:0/890399275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c1a8680 con 0x7f697c108670 2024-05-03T17:53:00.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.641+0000 7f6972ffd700 1 -- 172.21.15.122:0/890399275 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69740054d0 con 0x7f697c108670 2024-05-03T17:53:00.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.641+0000 7f6983f64700 1 -- 172.21.15.122:0/890399275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c1a8c10 con 0x7f697c108670 2024-05-03T17:53:00.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.641+0000 7f6972ffd700 1 -- 172.21.15.122:0/890399275 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974007bf0 con 0x7f697c108670 2024-05-03T17:53:00.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.642+0000 7f6983f64700 1 -- 172.21.15.122:0/890399275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c06a190 con 0x7f697c108670 2024-05-03T17:53:00.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.643+0000 7f6972ffd700 1 -- 172.21.15.122:0/890399275 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f6974004fb0 con 0x7f697c108670 2024-05-03T17:53:00.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.643+0000 7f6972ffd700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f69680b2880 0x7f69680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:00.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.644+0000 7f6981d00700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f69680b2880 0x7f69680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:00.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.644+0000 7f6972ffd700 1 -- 172.21.15.122:0/890399275 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f69740e1ed0 con 0x7f697c108670 2024-05-03T17:53:00.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.644+0000 7f6981d00700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f69680b2880 0x7f69680b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f697c06b600 tx=0x7f697800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:00.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.647+0000 7f6972ffd700 1 -- 172.21.15.122:0/890399275 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6974014030 con 0x7f697c108670 2024-05-03T17:53:00.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.880+0000 7f6983f64700 1 -- 172.21.15.122:0/890399275 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f697c06b9f0 con 0x7f69680b2880 2024-05-03T17:53:00.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.910+0000 7f6972ffd700 1 -- 172.21.15.122:0/890399275 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f697c06b9f0 con 0x7f69680b2880 2024-05-03T17:53:00.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 -- 172.21.15.122:0/890399275 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f69680b2880 msgr2=0x7f69680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:00.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f69680b2880 0x7f69680b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f697c06b600 tx=0x7f697800d040 comp rx=0 tx=0).stop 2024-05-03T17:53:00.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 -- 172.21.15.122:0/890399275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c108670 msgr2=0x7f697c1995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:00.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c108670 0x7f697c1995a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f697400c970 tx=0x7f697400cc80 comp rx=0 tx=0).stop 2024-05-03T17:53:00.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 -- 172.21.15.122:0/890399275 shutdown_connections 2024-05-03T17:53:00.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f69680b2880 0x7f69680b4d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:00.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c108670 0x7f697c1995a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:00.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c102b00 0x7f697c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:00.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 --2- 172.21.15.122:0/890399275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f697c1034c0 0x7f697c199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:00.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 -- 172.21.15.122:0/890399275 >> 172.21.15.122:0/890399275 conn(0x7f697c0789a0 msgr2=0x7f697c100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:00.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 -- 172.21.15.122:0/890399275 shutdown_connections 2024-05-03T17:53:00.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:00.912+0000 7f6983f64700 1 -- 172.21.15.122:0/890399275 wait complete. 2024-05-03T17:53:01.210 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:53:00 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:53:00.884+0000 7f3742eaa700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T17:53:01.210 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:53:00 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:53:00.884+0000 7f3742eaa700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T17:53:01.210 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:53:00 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:53:00.884+0000 7f3742eaa700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T17:53:01.210 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:53:00 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:53:00.884+0000 7f3742eaa700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T17:53:01.210 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 17:53:00 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x[29973]: 2024-05-03T17:53:00.885+0000 7f3742eaa700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T17:53:01.655 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-03T17:53:01.899 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:01.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:01 smithi122 ceph-mon[29744]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-03T17:53:01.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:01 smithi122 ceph-mon[29744]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:02.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:01 smithi192 ceph-mon[33753]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-03T17:53:02.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:01 smithi192 ceph-mon[33753]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:02.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:01 smithi149 ceph-mon[33409]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-03T17:53:02.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:01 smithi149 ceph-mon[33409]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:03.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.582+0000 7f0e94cc2700 1 -- 172.21.15.122:0/2474830346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e9006bc70 msgr2=0x7f0e9006c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:03.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.582+0000 7f0e94cc2700 1 --2- 172.21.15.122:0/2474830346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e9006bc70 0x7f0e9006c0f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0e780099c0 tx=0x7f0e78009cd0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.582+0000 7f0e94cc2700 1 -- 172.21.15.122:0/2474830346 shutdown_connections 2024-05-03T17:53:03.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.582+0000 7f0e94cc2700 1 --2- 172.21.15.122:0/2474830346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e9006c630 0x7f0e90110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.582+0000 7f0e94cc2700 1 --2- 172.21.15.122:0/2474830346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e901095c0 0x7f0e901099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.582+0000 7f0e94cc2700 1 --2- 172.21.15.122:0/2474830346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e9006bc70 0x7f0e9006c0f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.582+0000 7f0e94cc2700 1 -- 172.21.15.122:0/2474830346 >> 172.21.15.122:0/2474830346 conn(0x7f0e900782b0 msgr2=0x7f0e900786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:03.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.582+0000 7f0e94cc2700 1 -- 172.21.15.122:0/2474830346 shutdown_connections 2024-05-03T17:53:03.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.582+0000 7f0e94cc2700 1 -- 172.21.15.122:0/2474830346 wait complete. 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e94cc2700 1 Processor -- start 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e94cc2700 1 -- start start 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e94cc2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e9006bc70 0x7f0e901025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e94cc2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e9006c630 0x7f0e90102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e94cc2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e901095c0 0x7f0e90103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e94cc2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901136d0 con 0x7f0e9006c630 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e94cc2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90113550 con 0x7f0e9006bc70 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e94cc2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90113850 con 0x7f0e901095c0 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e8dd9b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e9006c630 0x7f0e90102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e8dd9b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e9006c630 0x7f0e90102ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53866/0 (socket says 172.21.15.122:53866) 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e8dd9b700 1 -- 172.21.15.122:0/937781258 learned_addr learned my addr 172.21.15.122:0/937781258 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e8ed9d700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e901095c0 0x7f0e90103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.583+0000 7f0e8e59c700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e9006bc70 0x7f0e901025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.584+0000 7f0e8dd9b700 1 -- 172.21.15.122:0/937781258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e901095c0 msgr2=0x7f0e90103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.584+0000 7f0e8dd9b700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e901095c0 0x7f0e90103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.584+0000 7f0e8dd9b700 1 -- 172.21.15.122:0/937781258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e9006bc70 msgr2=0x7f0e901025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.584+0000 7f0e8dd9b700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e9006bc70 0x7f0e901025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.584+0000 7f0e8dd9b700 1 -- 172.21.15.122:0/937781258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e901a8110 con 0x7f0e9006c630 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.584+0000 7f0e8dd9b700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e9006c630 0x7f0e90102ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800b580 tx=0x7f0e780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.584+0000 7f0e877fe700 1 -- 172.21.15.122:0/937781258 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78027070 con 0x7f0e9006c630 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.584+0000 7f0e877fe700 1 -- 172.21.15.122:0/937781258 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e78003f20 con 0x7f0e9006c630 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.584+0000 7f0e94cc2700 1 -- 172.21.15.122:0/937781258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e901a83a0 con 0x7f0e9006c630 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.584+0000 7f0e877fe700 1 -- 172.21.15.122:0/937781258 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78022d10 con 0x7f0e9006c630 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.584+0000 7f0e94cc2700 1 -- 172.21.15.122:0/937781258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e901a8850 con 0x7f0e9006c630 2024-05-03T17:53:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.585+0000 7f0e94cc2700 1 -- 172.21.15.122:0/937781258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9010ba50 con 0x7f0e9006c630 2024-05-03T17:53:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.586+0000 7f0e877fe700 1 -- 172.21.15.122:0/937781258 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f0e78022470 con 0x7f0e9006c630 2024-05-03T17:53:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.587+0000 7f0e877fe700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0e7c0b2990 0x7f0e7c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.587+0000 7f0e8e59c700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0e7c0b2990 0x7f0e7c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.587+0000 7f0e877fe700 1 -- 172.21.15.122:0/937781258 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f0e780e3120 con 0x7f0e9006c630 2024-05-03T17:53:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.587+0000 7f0e8e59c700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0e7c0b2990 0x7f0e7c0b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e800045b0 tx=0x7f0e8000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:03.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.590+0000 7f0e877fe700 1 -- 172.21.15.122:0/937781258 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0e78013070 con 0x7f0e9006c630 2024-05-03T17:53:03.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.828+0000 7f0e94cc2700 1 -- 172.21.15.122:0/937781258 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0e900638e0 con 0x7f0e7c0b2990 2024-05-03T17:53:03.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.857+0000 7f0e877fe700 1 -- 172.21.15.122:0/937781258 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0e900638e0 con 0x7f0e7c0b2990 2024-05-03T17:53:03.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 -- 172.21.15.122:0/937781258 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0e7c0b2990 msgr2=0x7f0e7c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:03.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0e7c0b2990 0x7f0e7c0b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e800045b0 tx=0x7f0e8000b3c0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 -- 172.21.15.122:0/937781258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e9006c630 msgr2=0x7f0e90102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e9006c630 0x7f0e90102ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800b580 tx=0x7f0e780047b0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 -- 172.21.15.122:0/937781258 shutdown_connections 2024-05-03T17:53:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0e7c0b2990 0x7f0e7c0b4e50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e901095c0 0x7f0e90103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e9006bc70 0x7f0e901025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 --2- 172.21.15.122:0/937781258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e9006c630 0x7f0e90102ae0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 -- 172.21.15.122:0/937781258 >> 172.21.15.122:0/937781258 conn(0x7f0e900782b0 msgr2=0x7f0e90101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 -- 172.21.15.122:0/937781258 shutdown_connections 2024-05-03T17:53:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:03.859+0000 7f0e94cc2700 1 -- 172.21.15.122:0/937781258 wait complete. 2024-05-03T17:53:04.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:03 smithi192 ceph-mon[33753]: pgmap v299: 97 pgs: 97 active+clean; 588 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-03T17:53:04.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:03 smithi192 ceph-mon[33753]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-05-03T17:53:04.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:03 smithi122 ceph-mon[29744]: pgmap v299: 97 pgs: 97 active+clean; 588 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-03T17:53:04.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:03 smithi122 ceph-mon[29744]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-05-03T17:53:04.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:03 smithi149 ceph-mon[33409]: pgmap v299: 97 pgs: 97 active+clean; 588 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-03T17:53:04.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:03 smithi149 ceph-mon[33409]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-05-03T17:53:04.658 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-03T17:53:04.911 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:04.953 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:04 smithi122 ceph-mon[29744]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:05.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:04 smithi192 ceph-mon[33753]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:05.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:04 smithi149 ceph-mon[33409]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:06.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:05 smithi122 ceph-mon[29744]: pgmap v300: 97 pgs: 97 active+clean; 589 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-03T17:53:06.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:05 smithi192 ceph-mon[33753]: pgmap v300: 97 pgs: 97 active+clean; 589 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-03T17:53:06.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:05 smithi149 ceph-mon[33409]: pgmap v300: 97 pgs: 97 active+clean; 589 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-03T17:53:06.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.609+0000 7f456ad92700 1 -- 172.21.15.122:0/3957953024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45641095e0 msgr2=0x7f45641099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:06.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.609+0000 7f456ad92700 1 --2- 172.21.15.122:0/3957953024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45641095e0 0x7f45641099c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f45580099c0 tx=0x7f4558009cd0 comp rx=0 tx=0).stop 2024-05-03T17:53:06.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.609+0000 7f456ad92700 1 -- 172.21.15.122:0/3957953024 shutdown_connections 2024-05-03T17:53:06.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.609+0000 7f456ad92700 1 --2- 172.21.15.122:0/3957953024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4564103f00 0x7f45641103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:06.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.609+0000 7f456ad92700 1 --2- 172.21.15.122:0/3957953024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45641095e0 0x7f45641099c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.609+0000 7f456ad92700 1 --2- 172.21.15.122:0/3957953024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4564103540 0x7f45641039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.609+0000 7f456ad92700 1 -- 172.21.15.122:0/3957953024 >> 172.21.15.122:0/3957953024 conn(0x7f456407a650 msgr2=0x7f456407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.609+0000 7f456ad92700 1 -- 172.21.15.122:0/3957953024 shutdown_connections 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.609+0000 7f456ad92700 1 -- 172.21.15.122:0/3957953024 wait complete. 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456ad92700 1 Processor -- start 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456ad92700 1 -- start start 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456ad92700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4564103540 0x7f456419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456ad92700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4564103f00 0x7f456419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456ad92700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45641095e0 0x7f45641a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456ad92700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4564113740 con 0x7f45641095e0 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456ad92700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f45641135c0 con 0x7f4564103540 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456ad92700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45641138c0 con 0x7f4564103f00 2024-05-03T17:53:06.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456932f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45641095e0 0x7f45641a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456932f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45641095e0 0x7f45641a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53888/0 (socket says 172.21.15.122:53888) 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456932f700 1 -- 172.21.15.122:0/22364437 learned_addr learned my addr 172.21.15.122:0/22364437 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f4568b2e700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4564103540 0x7f456419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f4563fff700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4564103f00 0x7f456419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456932f700 1 -- 172.21.15.122:0/22364437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4564103f00 msgr2=0x7f456419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456932f700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4564103f00 0x7f456419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456932f700 1 -- 172.21.15.122:0/22364437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4564103540 msgr2=0x7f456419bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456932f700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4564103540 0x7f456419bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456932f700 1 -- 172.21.15.122:0/22364437 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45641a0d90 con 0x7f45641095e0 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.610+0000 7f456932f700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45641095e0 0x7f45641a0650 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f455c00ada0 tx=0x7f455c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.611+0000 7f4561ffb700 1 -- 172.21.15.122:0/22364437 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c017020 con 0x7f45641095e0 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.611+0000 7f4561ffb700 1 -- 172.21.15.122:0/22364437 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f455c00ce30 con 0x7f45641095e0 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.611+0000 7f4561ffb700 1 -- 172.21.15.122:0/22364437 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c0054a0 con 0x7f45641095e0 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.611+0000 7f456ad92700 1 -- 172.21.15.122:0/22364437 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45641a1080 con 0x7f45641095e0 2024-05-03T17:53:06.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.611+0000 7f456ad92700 1 -- 172.21.15.122:0/22364437 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45641a8900 con 0x7f45641095e0 2024-05-03T17:53:06.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.612+0000 7f456ad92700 1 -- 172.21.15.122:0/22364437 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456406a190 con 0x7f45641095e0 2024-05-03T17:53:06.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.613+0000 7f4561ffb700 1 -- 172.21.15.122:0/22364437 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f455c00c980 con 0x7f45641095e0 2024-05-03T17:53:06.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.613+0000 7f4561ffb700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f45540b2910 0x7f45540b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:06.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.613+0000 7f4568b2e700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f45540b2910 0x7f45540b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:06.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.614+0000 7f4561ffb700 1 -- 172.21.15.122:0/22364437 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f455c0e19e0 con 0x7f45641095e0 2024-05-03T17:53:06.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.614+0000 7f4568b2e700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f45540b2910 0x7f45540b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4558009750 tx=0x7f455801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:06.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.618+0000 7f4561ffb700 1 -- 172.21.15.122:0/22364437 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f455c021020 con 0x7f45641095e0 2024-05-03T17:53:06.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.855+0000 7f456ad92700 1 -- 172.21.15.122:0/22364437 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f45640638e0 con 0x7f45540b2910 2024-05-03T17:53:06.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.861+0000 7f4561ffb700 1 -- 172.21.15.122:0/22364437 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f45640638e0 con 0x7f45540b2910 2024-05-03T17:53:06.856 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/qa/sv_0/13ba2ef8-1a02-4705-8523-c3203d62b1fa 2024-05-03T17:53:06.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.863+0000 7f456ad92700 1 -- 172.21.15.122:0/22364437 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f45540b2910 msgr2=0x7f45540b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:06.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.863+0000 7f456ad92700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f45540b2910 0x7f45540b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4558009750 tx=0x7f455801a040 comp rx=0 tx=0).stop 2024-05-03T17:53:06.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.863+0000 7f456ad92700 1 -- 172.21.15.122:0/22364437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45641095e0 msgr2=0x7f45641a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:06.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.863+0000 7f456ad92700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45641095e0 0x7f45641a0650 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f455c00ada0 tx=0x7f455c00c330 comp rx=0 tx=0).stop 2024-05-03T17:53:06.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.863+0000 7f456ad92700 1 -- 172.21.15.122:0/22364437 shutdown_connections 2024-05-03T17:53:06.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.863+0000 7f456ad92700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f45540b2910 0x7f45540b4dd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:06.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.863+0000 7f456ad92700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45641095e0 0x7f45641a0650 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:06.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.863+0000 7f456ad92700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4564103540 0x7f456419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:06.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.864+0000 7f456ad92700 1 --2- 172.21.15.122:0/22364437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4564103f00 0x7f456419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:06.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.864+0000 7f456ad92700 1 -- 172.21.15.122:0/22364437 >> 172.21.15.122:0/22364437 conn(0x7f456407a650 msgr2=0x7f456410ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:06.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.864+0000 7f456ad92700 1 -- 172.21.15.122:0/22364437 shutdown_connections 2024-05-03T17:53:06.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:06.864+0000 7f456ad92700 1 -- 172.21.15.122:0/22364437 wait complete. 2024-05-03T17:53:07.534 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-03T17:53:07.775 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:08.089 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:07 smithi149 ceph-mon[33409]: pgmap v301: 97 pgs: 97 active+clean; 589 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-03T17:53:08.089 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:07 smithi149 ceph-mon[33409]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:08.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:07 smithi192 ceph-mon[33753]: pgmap v301: 97 pgs: 97 active+clean; 589 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-03T17:53:08.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:07 smithi192 ceph-mon[33753]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:08.148 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:07 smithi122 ceph-mon[29744]: pgmap v301: 97 pgs: 97 active+clean; 589 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-03T17:53:08.148 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:07 smithi122 ceph-mon[29744]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.458+0000 7f672a296700 1 -- 172.21.15.122:0/1398330558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6724102d50 msgr2=0x7f672410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.458+0000 7f672a296700 1 --2- 172.21.15.122:0/1398330558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6724102d50 0x7f672410f240 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f671c00b0a0 tx=0x7f671c00b3b0 comp rx=0 tx=0).stop 2024-05-03T17:53:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.458+0000 7f672a296700 1 -- 172.21.15.122:0/1398330558 shutdown_connections 2024-05-03T17:53:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.458+0000 7f672a296700 1 --2- 172.21.15.122:0/1398330558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6724102d50 0x7f672410f240 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.458+0000 7f672a296700 1 --2- 172.21.15.122:0/1398330558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6724108400 0x7f67241087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.458+0000 7f672a296700 1 --2- 172.21.15.122:0/1398330558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6724102390 0x7f6724102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.458+0000 7f672a296700 1 -- 172.21.15.122:0/1398330558 >> 172.21.15.122:0/1398330558 conn(0x7f6724079310 msgr2=0x7f6724079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.458+0000 7f672a296700 1 -- 172.21.15.122:0/1398330558 shutdown_connections 2024-05-03T17:53:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.458+0000 7f672a296700 1 -- 172.21.15.122:0/1398330558 wait complete. 2024-05-03T17:53:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.459+0000 7f672a296700 1 Processor -- start 2024-05-03T17:53:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.459+0000 7f672a296700 1 -- start start 2024-05-03T17:53:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.459+0000 7f672a296700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6724102390 0x7f672419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.459+0000 7f672a296700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6724102d50 0x7f672419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.459+0000 7f672a296700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6724108400 0x7f67241987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.459+0000 7f672a296700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6724112550 con 0x7f6724102d50 2024-05-03T17:53:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.459+0000 7f672a296700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f67241123d0 con 0x7f6724108400 2024-05-03T17:53:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.459+0000 7f672a296700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67241126d0 con 0x7f6724102390 2024-05-03T17:53:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.459+0000 7f67237fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6724102d50 0x7f672419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.459+0000 7f67237fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6724102d50 0x7f672419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51174/0 (socket says 172.21.15.122:51174) 2024-05-03T17:53:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.459+0000 7f67237fe700 1 -- 172.21.15.122:0/1172385699 learned_addr learned my addr 172.21.15.122:0/1172385699 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f6728833700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6724108400 0x7f67241987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f6723fff700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6724102390 0x7f672419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f67237fe700 1 -- 172.21.15.122:0/1172385699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6724102390 msgr2=0x7f672419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f67237fe700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6724102390 0x7f672419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f67237fe700 1 -- 172.21.15.122:0/1172385699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6724108400 msgr2=0x7f67241987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f67237fe700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6724108400 0x7f67241987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f67237fe700 1 -- 172.21.15.122:0/1172385699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6724198fb0 con 0x7f6724102d50 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f6728833700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6724108400 0x7f67241987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f67237fe700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6724102d50 0x7f672419eb90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f671800d980 tx=0x7f671800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f67217fa700 1 -- 172.21.15.122:0/1172385699 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718017070 con 0x7f6724102d50 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f672a296700 1 -- 172.21.15.122:0/1172385699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67241992a0 con 0x7f6724102d50 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f67217fa700 1 -- 172.21.15.122:0/1172385699 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6718004830 con 0x7f6724102d50 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f67217fa700 1 -- 172.21.15.122:0/1172385699 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718005440 con 0x7f6724102d50 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.460+0000 7f672a296700 1 -- 172.21.15.122:0/1172385699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67241a54b0 con 0x7f6724102d50 2024-05-03T17:53:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.461+0000 7f672a296700 1 -- 172.21.15.122:0/1172385699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6724103ab0 con 0x7f6724102d50 2024-05-03T17:53:09.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.463+0000 7f67217fa700 1 -- 172.21.15.122:0/1172385699 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f67180055e0 con 0x7f6724102d50 2024-05-03T17:53:09.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.464+0000 7f67217fa700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f670c0b2910 0x7f670c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.464+0000 7f6723fff700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f670c0b2910 0x7f670c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.464+0000 7f67217fa700 1 -- 172.21.15.122:0/1172385699 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f67180e2570 con 0x7f6724102d50 2024-05-03T17:53:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.464+0000 7f6723fff700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f670c0b2910 0x7f670c0b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6714005fd0 tx=0x7f6714005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:09.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.467+0000 7f67217fa700 1 -- 172.21.15.122:0/1172385699 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6718020020 con 0x7f6724102d50 2024-05-03T17:53:09.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.706+0000 7f672a296700 1 -- 172.21.15.122:0/1172385699 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f67240638e0 con 0x7f670c0b2910 2024-05-03T17:53:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.731+0000 7f67217fa700 1 -- 172.21.15.122:0/1172385699 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f67240638e0 con 0x7f670c0b2910 2024-05-03T17:53:09.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.733+0000 7f672a296700 1 -- 172.21.15.122:0/1172385699 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f670c0b2910 msgr2=0x7f670c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:09.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.733+0000 7f672a296700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f670c0b2910 0x7f670c0b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6714005fd0 tx=0x7f6714005db0 comp rx=0 tx=0).stop 2024-05-03T17:53:09.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.733+0000 7f672a296700 1 -- 172.21.15.122:0/1172385699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6724102d50 msgr2=0x7f672419eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:09.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.733+0000 7f672a296700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6724102d50 0x7f672419eb90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f671800d980 tx=0x7f671800dd40 comp rx=0 tx=0).stop 2024-05-03T17:53:09.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.734+0000 7f672a296700 1 -- 172.21.15.122:0/1172385699 shutdown_connections 2024-05-03T17:53:09.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.734+0000 7f672a296700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f670c0b2910 0x7f670c0b4dd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:09.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.734+0000 7f672a296700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6724108400 0x7f67241987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:09.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.734+0000 7f672a296700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6724102390 0x7f672419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:09.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.734+0000 7f672a296700 1 --2- 172.21.15.122:0/1172385699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6724102d50 0x7f672419eb90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:09.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.734+0000 7f672a296700 1 -- 172.21.15.122:0/1172385699 >> 172.21.15.122:0/1172385699 conn(0x7f6724079310 msgr2=0x7f6724100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:09.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.734+0000 7f672a296700 1 -- 172.21.15.122:0/1172385699 shutdown_connections 2024-05-03T17:53:09.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:09.734+0000 7f672a296700 1 -- 172.21.15.122:0/1172385699 wait complete. 2024-05-03T17:53:09.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:09 smithi122 ceph-mon[29744]: pgmap v302: 97 pgs: 97 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-03T17:53:10.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:09 smithi192 ceph-mon[33753]: pgmap v302: 97 pgs: 97 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-03T17:53:10.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:09 smithi149 ceph-mon[33409]: pgmap v302: 97 pgs: 97 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-03T17:53:10.366 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-03T17:53:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:11.058 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:10 smithi122 ceph-mon[29744]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:11.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:10 smithi192 ceph-mon[33753]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:11.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:10 smithi149 ceph-mon[33409]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:11.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:11 smithi122 ceph-mon[29744]: pgmap v303: 97 pgs: 97 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-03T17:53:12.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:11 smithi192 ceph-mon[33753]: pgmap v303: 97 pgs: 97 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-03T17:53:12.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.234+0000 7f2765d5f700 1 -- 172.21.15.122:0/2768148329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2760107bf0 msgr2=0x7f2760107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:12.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.234+0000 7f2765d5f700 1 --2- 172.21.15.122:0/2768148329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2760107bf0 0x7f2760107fd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f27500099c0 tx=0x7f2750009cd0 comp rx=0 tx=0).stop 2024-05-03T17:53:12.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.234+0000 7f2765d5f700 1 -- 172.21.15.122:0/2768148329 shutdown_connections 2024-05-03T17:53:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.234+0000 7f2765d5f700 1 --2- 172.21.15.122:0/2768148329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2760102500 0x7f276010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.234+0000 7f2765d5f700 1 --2- 172.21.15.122:0/2768148329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2760107bf0 0x7f2760107fd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.234+0000 7f2765d5f700 1 --2- 172.21.15.122:0/2768148329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2760101b40 0x7f2760101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.234+0000 7f2765d5f700 1 -- 172.21.15.122:0/2768148329 >> 172.21.15.122:0/2768148329 conn(0x7f27600789a0 msgr2=0x7f27600ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.234+0000 7f2765d5f700 1 -- 172.21.15.122:0/2768148329 shutdown_connections 2024-05-03T17:53:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.234+0000 7f2765d5f700 1 -- 172.21.15.122:0/2768148329 wait complete. 2024-05-03T17:53:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.235+0000 7f2765d5f700 1 Processor -- start 2024-05-03T17:53:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.235+0000 7f2765d5f700 1 -- start start 2024-05-03T17:53:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.235+0000 7f2765d5f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2760101b40 0x7f27601037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.235+0000 7f2765d5f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2760102500 0x7f2760103cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.235+0000 7f2765d5f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2760107bf0 0x7f2760104230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.235+0000 7f2765d5f700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2760112650 con 0x7f2760102500 2024-05-03T17:53:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.235+0000 7f2765d5f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f27601124d0 con 0x7f2760107bf0 2024-05-03T17:53:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.235+0000 7f2765d5f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f27601127d0 con 0x7f2760101b40 2024-05-03T17:53:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.235+0000 7f275effd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2760102500 0x7f2760103cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.235+0000 7f275effd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2760102500 0x7f2760103cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51188/0 (socket says 172.21.15.122:51188) 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.235+0000 7f275effd700 1 -- 172.21.15.122:0/943203452 learned_addr learned my addr 172.21.15.122:0/943203452 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f275ffff700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2760107bf0 0x7f2760104230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f275f7fe700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2760101b40 0x7f27601037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f275effd700 1 -- 172.21.15.122:0/943203452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2760101b40 msgr2=0x7f27601037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f275effd700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2760101b40 0x7f27601037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f275effd700 1 -- 172.21.15.122:0/943203452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2760107bf0 msgr2=0x7f2760104230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f275effd700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2760107bf0 0x7f2760104230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f275effd700 1 -- 172.21.15.122:0/943203452 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27601a8100 con 0x7f2760102500 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f275effd700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2760102500 0x7f2760103cf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f275400b530 tx=0x7f275400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f275cff9700 1 -- 172.21.15.122:0/943203452 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754015020 con 0x7f2760102500 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f2765d5f700 1 -- 172.21.15.122:0/943203452 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27601a83f0 con 0x7f2760102500 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f275cff9700 1 -- 172.21.15.122:0/943203452 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2754011ce0 con 0x7f2760102500 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f2765d5f700 1 -- 172.21.15.122:0/943203452 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27601a8980 con 0x7f2760102500 2024-05-03T17:53:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.236+0000 7f275cff9700 1 -- 172.21.15.122:0/943203452 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754010e20 con 0x7f2760102500 2024-05-03T17:53:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.238+0000 7f2765d5f700 1 -- 172.21.15.122:0/943203452 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276006a190 con 0x7f2760102500 2024-05-03T17:53:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.239+0000 7f275cff9700 1 -- 172.21.15.122:0/943203452 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2754018020 con 0x7f2760102500 2024-05-03T17:53:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.239+0000 7f275cff9700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f27480b2910 0x7f27480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.239+0000 7f275f7fe700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f27480b2910 0x7f27480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.239+0000 7f275cff9700 1 -- 172.21.15.122:0/943203452 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f27540e1900 con 0x7f2760102500 2024-05-03T17:53:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.240+0000 7f275f7fe700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f27480b2910 0x7f27480b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2750009750 tx=0x7f275001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:12.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.244+0000 7f275cff9700 1 -- 172.21.15.122:0/943203452 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f275401f020 con 0x7f2760102500 2024-05-03T17:53:12.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:11 smithi149 ceph-mon[33409]: pgmap v303: 97 pgs: 97 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-03T17:53:12.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.481+0000 7f2765d5f700 1 -- 172.21.15.122:0/943203452 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f276010f3c0 con 0x7f27480b2910 2024-05-03T17:53:12.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.486+0000 7f275cff9700 1 -- 172.21.15.122:0/943203452 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f276010f3c0 con 0x7f27480b2910 2024-05-03T17:53:12.481 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/qa/sv_1/c64844e0-c7be-4cc0-80e5-d69d525a340f 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 -- 172.21.15.122:0/943203452 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f27480b2910 msgr2=0x7f27480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f27480b2910 0x7f27480b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2750009750 tx=0x7f275001a040 comp rx=0 tx=0).stop 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 -- 172.21.15.122:0/943203452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2760102500 msgr2=0x7f2760103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2760102500 0x7f2760103cf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f275400b530 tx=0x7f275400b840 comp rx=0 tx=0).stop 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 -- 172.21.15.122:0/943203452 shutdown_connections 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f27480b2910 0x7f27480b4dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2760107bf0 0x7f2760104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2760101b40 0x7f27601037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 --2- 172.21.15.122:0/943203452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2760102500 0x7f2760103cf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 -- 172.21.15.122:0/943203452 >> 172.21.15.122:0/943203452 conn(0x7f27600789a0 msgr2=0x7f2760100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 -- 172.21.15.122:0/943203452 shutdown_connections 2024-05-03T17:53:12.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:12.489+0000 7f2765d5f700 1 -- 172.21.15.122:0/943203452 wait complete. 2024-05-03T17:53:13.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:12 smithi192 ceph-mon[33753]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:13.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:12 smithi122 ceph-mon[29744]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:13.149 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:53:13.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:12 smithi149 ceph-mon[33409]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:53:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:14.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:13 smithi192 ceph-mon[33753]: pgmap v304: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-05-03T17:53:14.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:13 smithi122 ceph-mon[29744]: pgmap v304: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-05-03T17:53:14.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:13 smithi149 ceph-mon[33409]: pgmap v304: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-05-03T17:53:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.175+0000 7f8292f55700 1 -- 172.21.15.122:0/3363421498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f828c103750 msgr2=0x7f828c103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.175+0000 7f8292f55700 1 --2- 172.21.15.122:0/3363421498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f828c103750 0x7f828c103b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82800099c0 tx=0x7f8280009cd0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.175+0000 7f8292f55700 1 -- 172.21.15.122:0/3363421498 shutdown_connections 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.175+0000 7f8292f55700 1 --2- 172.21.15.122:0/3363421498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f828c108850 0x7f828c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.175+0000 7f8292f55700 1 --2- 172.21.15.122:0/3363421498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f828c103750 0x7f828c103b30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.175+0000 7f8292f55700 1 --2- 172.21.15.122:0/3363421498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f828c104100 0x7f828c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.175+0000 7f8292f55700 1 -- 172.21.15.122:0/3363421498 >> 172.21.15.122:0/3363421498 conn(0x7f828c07a600 msgr2=0x7f828c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.175+0000 7f8292f55700 1 -- 172.21.15.122:0/3363421498 shutdown_connections 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.175+0000 7f8292f55700 1 -- 172.21.15.122:0/3363421498 wait complete. 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f8292f55700 1 Processor -- start 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f8292f55700 1 -- start start 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f8292f55700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f828c103750 0x7f828c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f8292f55700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f828c104100 0x7f828c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f8292f55700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f828c108850 0x7f828c076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f8292f55700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f828c113620 con 0x7f828c103750 2024-05-03T17:53:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f8292f55700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1134a0 con 0x7f828c104100 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f8292f55700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1137a0 con 0x7f828c108850 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f8290cf1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f828c103750 0x7f828c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f8290cf1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f828c103750 0x7f828c0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51200/0 (socket says 172.21.15.122:51200) 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f8290cf1700 1 -- 172.21.15.122:0/3107135655 learned_addr learned my addr 172.21.15.122:0/3107135655 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.176+0000 7f828bfff700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f828c104100 0x7f828c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.177+0000 7f8290cf1700 1 -- 172.21.15.122:0/3107135655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f828c108850 msgr2=0x7f828c076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.177+0000 7f8290cf1700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f828c108850 0x7f828c076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.177+0000 7f8290cf1700 1 -- 172.21.15.122:0/3107135655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f828c104100 msgr2=0x7f828c075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.177+0000 7f8290cf1700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f828c104100 0x7f828c075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.177+0000 7f8290cf1700 1 -- 172.21.15.122:0/3107135655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828c1a8020 con 0x7f828c103750 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.177+0000 7f8290cf1700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f828c103750 0x7f828c0757a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f828000b580 tx=0x7f82800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.177+0000 7f8289ffb700 1 -- 172.21.15.122:0/3107135655 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280027070 con 0x7f828c103750 2024-05-03T17:53:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.177+0000 7f8292f55700 1 -- 172.21.15.122:0/3107135655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828c1a82b0 con 0x7f828c103750 2024-05-03T17:53:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.177+0000 7f8289ffb700 1 -- 172.21.15.122:0/3107135655 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8280003f20 con 0x7f828c103750 2024-05-03T17:53:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.177+0000 7f8289ffb700 1 -- 172.21.15.122:0/3107135655 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280022d70 con 0x7f828c103750 2024-05-03T17:53:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.177+0000 7f8292f55700 1 -- 172.21.15.122:0/3107135655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f828c1a8760 con 0x7f828c103750 2024-05-03T17:53:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.178+0000 7f8292f55700 1 -- 172.21.15.122:0/3107135655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f828c10ba90 con 0x7f828c103750 2024-05-03T17:53:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.180+0000 7f8289ffb700 1 -- 172.21.15.122:0/3107135655 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8280022500 con 0x7f828c103750 2024-05-03T17:53:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.180+0000 7f8289ffb700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f827c0b2910 0x7f827c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.180+0000 7f828bfff700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f827c0b2910 0x7f827c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.180+0000 7f8289ffb700 1 -- 172.21.15.122:0/3107135655 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f82800e3280 con 0x7f828c103750 2024-05-03T17:53:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.181+0000 7f828bfff700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f827c0b2910 0x7f827c0b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f828c1030a0 tx=0x7f82780073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.184+0000 7f8289ffb700 1 -- 172.21.15.122:0/3107135655 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8280013070 con 0x7f828c103750 2024-05-03T17:53:15.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.438+0000 7f8292f55700 1 -- 172.21.15.122:0/3107135655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f828c02e510 con 0x7f828c103750 2024-05-03T17:53:15.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.439+0000 7f8289ffb700 1 -- 172.21.15.122:0/3107135655 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11892 (secure 0 0 0) 0x7f82800ab080 con 0x7f828c103750 2024-05-03T17:53:15.434 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:53:15.434 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/555236113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":555236113},{"type":"v1","addr":"172.21.15.122:6839","nonce":555236113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1783292878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1783292878},{"type":"v1","addr":"172.21.15.122:6841","nonce":1783292878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T17:52:37.616728+0000","modified":"2024-05-03T17:52:56.867078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24463,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24410":{"gid":24410,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1529632960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1529632960},{"type":"v1","addr":"172.21.15.149:6839","nonce":1529632960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1429818440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1429818440},{"type":"v1","addr":"172.21.15.192:6839","nonce":1429818440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24457,24463]},"id":1}]} 2024-05-03T17:53:15.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.441+0000 7f8292f55700 1 -- 172.21.15.122:0/3107135655 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f827c0b2910 msgr2=0x7f827c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.441+0000 7f8292f55700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f827c0b2910 0x7f827c0b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f828c1030a0 tx=0x7f82780073d0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.441+0000 7f8292f55700 1 -- 172.21.15.122:0/3107135655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f828c103750 msgr2=0x7f828c0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.441+0000 7f8292f55700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f828c103750 0x7f828c0757a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f828000b580 tx=0x7f82800047b0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.441+0000 7f8292f55700 1 -- 172.21.15.122:0/3107135655 shutdown_connections 2024-05-03T17:53:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.441+0000 7f8292f55700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f827c0b2910 0x7f827c0b4dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.441+0000 7f8292f55700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f828c108850 0x7f828c076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.441+0000 7f8292f55700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f828c103750 0x7f828c0757a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.441+0000 7f8292f55700 1 --2- 172.21.15.122:0/3107135655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f828c104100 0x7f828c075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.441+0000 7f8292f55700 1 -- 172.21.15.122:0/3107135655 >> 172.21.15.122:0/3107135655 conn(0x7f828c07a600 msgr2=0x7f828c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.442+0000 7f8292f55700 1 -- 172.21.15.122:0/3107135655 shutdown_connections 2024-05-03T17:53:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:15.442+0000 7f8292f55700 1 -- 172.21.15.122:0/3107135655 wait complete. 2024-05-03T17:53:15.438 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 24 2024-05-03T17:53:16.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:15 smithi122 ceph-mon[29744]: pgmap v305: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-03T17:53:16.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:15 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3107135655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:53:16.110 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T17:53:16.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:15 smithi192 ceph-mon[33753]: pgmap v305: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-03T17:53:16.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:15 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3107135655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:53:16.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:15 smithi149 ceph-mon[33409]: pgmap v305: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-03T17:53:16.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:15 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3107135655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:53:16.348 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:18.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.024+0000 7fb652d8e700 1 -- 172.21.15.122:0/3830798999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb64c1019f0 msgr2=0x7fb64c101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:18.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.024+0000 7fb652d8e700 1 --2- 172.21.15.122:0/3830798999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb64c1019f0 0x7fb64c101e70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb638009990 tx=0x7fb638009ca0 comp rx=0 tx=0).stop 2024-05-03T17:53:18.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.024+0000 7fb652d8e700 1 -- 172.21.15.122:0/3830798999 shutdown_connections 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.024+0000 7fb652d8e700 1 --2- 172.21.15.122:0/3830798999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb64c102490 0x7fb64c106130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.024+0000 7fb652d8e700 1 --2- 172.21.15.122:0/3830798999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c108620 0x7fb64c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.024+0000 7fb652d8e700 1 --2- 172.21.15.122:0/3830798999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb64c1019f0 0x7fb64c101e70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.024+0000 7fb652d8e700 1 -- 172.21.15.122:0/3830798999 >> 172.21.15.122:0/3830798999 conn(0x7fb64c0789a0 msgr2=0x7fb64c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.024+0000 7fb652d8e700 1 -- 172.21.15.122:0/3830798999 shutdown_connections 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.024+0000 7fb652d8e700 1 -- 172.21.15.122:0/3830798999 wait complete. 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb652d8e700 1 Processor -- start 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb652d8e700 1 -- start start 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb652d8e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb64c1019f0 0x7fb64c1a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb652d8e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb64c102490 0x7fb64c1a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb652d8e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c108620 0x7fb64c1a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:18.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb652d8e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c119f90 con 0x7fb64c1019f0 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb652d8e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c119e10 con 0x7fb64c108620 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb652d8e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c11a110 con 0x7fb64c102490 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb650b2a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb64c1019f0 0x7fb64c1a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb650b2a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb64c1019f0 0x7fb64c1a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51224/0 (socket says 172.21.15.122:51224) 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb650b2a700 1 -- 172.21.15.122:0/3467259492 learned_addr learned my addr 172.21.15.122:0/3467259492 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb64bfff700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb64c102490 0x7fb64c1a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.025+0000 7fb65132b700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c108620 0x7fb64c1a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.026+0000 7fb650b2a700 1 -- 172.21.15.122:0/3467259492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb64c102490 msgr2=0x7fb64c1a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.026+0000 7fb650b2a700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb64c102490 0x7fb64c1a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.026+0000 7fb650b2a700 1 -- 172.21.15.122:0/3467259492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c108620 msgr2=0x7fb64c1a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.026+0000 7fb650b2a700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c108620 0x7fb64c1a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.026+0000 7fb650b2a700 1 -- 172.21.15.122:0/3467259492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c1a54d0 con 0x7fb64c1019f0 2024-05-03T17:53:18.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.026+0000 7fb650b2a700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb64c1019f0 0x7fb64c1a04c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb64000b530 tx=0x7fb64000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:18.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.026+0000 7fb649ffb700 1 -- 172.21.15.122:0/3467259492 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640015020 con 0x7fb64c1019f0 2024-05-03T17:53:18.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.026+0000 7fb652d8e700 1 -- 172.21.15.122:0/3467259492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64c1a57c0 con 0x7fb64c1019f0 2024-05-03T17:53:18.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.026+0000 7fb649ffb700 1 -- 172.21.15.122:0/3467259492 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb640011ce0 con 0x7fb64c1019f0 2024-05-03T17:53:18.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.026+0000 7fb649ffb700 1 -- 172.21.15.122:0/3467259492 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640010e40 con 0x7fb64c1019f0 2024-05-03T17:53:18.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.026+0000 7fb652d8e700 1 -- 172.21.15.122:0/3467259492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64c1ad040 con 0x7fb64c1019f0 2024-05-03T17:53:18.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.027+0000 7fb652d8e700 1 -- 172.21.15.122:0/3467259492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64c06a190 con 0x7fb64c1019f0 2024-05-03T17:53:18.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.028+0000 7fb649ffb700 1 -- 172.21.15.122:0/3467259492 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb640018020 con 0x7fb64c1019f0 2024-05-03T17:53:18.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.029+0000 7fb649ffb700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb63c0b2910 0x7fb63c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:18.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.029+0000 7fb64bfff700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb63c0b2910 0x7fb63c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:18.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.029+0000 7fb649ffb700 1 -- 172.21.15.122:0/3467259492 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fb6400e1910 con 0x7fb64c1019f0 2024-05-03T17:53:18.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.029+0000 7fb64bfff700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb63c0b2910 0x7fb63c0b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb63800b5c0 tx=0x7fb638005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:18.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.033+0000 7fb649ffb700 1 -- 172.21.15.122:0/3467259492 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb64001f020 con 0x7fb64c1019f0 2024-05-03T17:53:18.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:17 smithi192 ceph-mon[33753]: pgmap v306: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-03T17:53:18.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:17 smithi122 ceph-mon[29744]: pgmap v306: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-03T17:53:18.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.245+0000 7fb652d8e700 1 -- 172.21.15.122:0/3467259492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb64c06b9f0 con 0x7fb64c1019f0 2024-05-03T17:53:18.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.246+0000 7fb649ffb700 1 -- 172.21.15.122:0/3467259492 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22283 (secure 0 0 0) 0x7fb6400a9790 con 0x7fb64c1019f0 2024-05-03T17:53:18.241 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:53:18.242 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":77,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","created":"2024-05-03T17:44:17.415571+0000","modified":"2024-05-03T17:52:37.616764+0000","last_up_change":"2024-05-03T17:51:28.839117+0000","last_in_change":"2024-05-03T17:51:07.954070+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T17:47:40.539967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T17:52:31.646919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T17:52:34.503067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4b08628-0a29-44aa-b8b9-ed9657f12be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6803","nonce":1329364610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6805","nonce":1329364610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6809","nonce":1329364610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6807","nonce":1329364610}]},"public_addr":"172.21.15.122:6803/1329364610","cluster_addr":"172.21.15.122:6805/1329364610","heartbeat_back_addr":"172.21.15.122:6809/1329364610","heartbeat_front_addr":"172.21.15.122:6807/1329364610","state":["exists","up"]},{"osd":1,"uuid":"ba550787-ce76-4944-8ebd-97e19590bb3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6801","nonce":1402463302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6803","nonce":1402463302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6807","nonce":1402463302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6805","nonce":1402463302}]},"public_addr":"172.21.15.149:6801/1402463302","cluster_addr":"172.21.15.149:6803/1402463302","heartbeat_back_addr":"172.21.15.149:6807/1402463302","heartbeat_front_addr":"172.21.15.149:6805/1402463302","state":["exists","up"]},{"osd":2,"uuid":"35ae7c47-d710-47c5-b0ec-91fa12b5be07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6801","nonce":3757191110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6803","nonce":3757191110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6807","nonce":3757191110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6805","nonce":3757191110}]},"public_addr":"172.21.15.192:6801/3757191110","cluster_addr":"172.21.15.192:6803/3757191110","heartbeat_back_addr":"172.21.15.192:6807/3757191110","heartbeat_front_addr":"172.21.15.192:6805/3757191110","state":["exists","up"]},{"osd":3,"uuid":"9973c0c1-ae73-497a-845e-664a20666df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6811","nonce":2359143764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6813","nonce":2359143764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6817","nonce":2359143764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6815","nonce":2359143764}]},"public_addr":"172.21.15.122:6811/2359143764","cluster_addr":"172.21.15.122:6813/2359143764","heartbeat_back_addr":"172.21.15.122:6817/2359143764","heartbeat_front_addr":"172.21.15.122:6815/2359143764","state":["exists","up"]},{"osd":4,"uuid":"da71dc18-94b9-4f7d-a061-7cc7b77db282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3627037532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3627037532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3627037532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3627037532}]},"public_addr":"172.21.15.149:6809/3627037532","cluster_addr":"172.21.15.149:6811/3627037532","heartbeat_back_addr":"172.21.15.149:6815/3627037532","heartbeat_front_addr":"172.21.15.149:6813/3627037532","state":["exists","up"]},{"osd":5,"uuid":"bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6809","nonce":455779485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6811","nonce":455779485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6815","nonce":455779485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6813","nonce":455779485}]},"public_addr":"172.21.15.192:6809/455779485","cluster_addr":"172.21.15.192:6811/455779485","heartbeat_back_addr":"172.21.15.192:6815/455779485","heartbeat_front_addr":"172.21.15.192:6813/455779485","state":["exists","up"]},{"osd":6,"uuid":"4150bfe5-a0e4-4bae-914a-75598f017519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6819","nonce":3419421998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6821","nonce":3419421998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6825","nonce":3419421998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6823","nonce":3419421998}]},"public_addr":"172.21.15.122:6819/3419421998","cluster_addr":"172.21.15.122:6821/3419421998","heartbeat_back_addr":"172.21.15.122:6825/3419421998","heartbeat_front_addr":"172.21.15.122:6823/3419421998","state":["exists","up"]},{"osd":7,"uuid":"71e01812-e511-44a0-a143-2afca510ae8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6817","nonce":139747549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6819","nonce":139747549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6823","nonce":139747549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6821","nonce":139747549}]},"public_addr":"172.21.15.149:6817/139747549","cluster_addr":"172.21.15.149:6819/139747549","heartbeat_back_addr":"172.21.15.149:6823/139747549","heartbeat_front_addr":"172.21.15.149:6821/139747549","state":["exists","up"]},{"osd":8,"uuid":"5294516d-fac6-41d0-a635-6bc1d7eca585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6817","nonce":2080816744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6819","nonce":2080816744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6823","nonce":2080816744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6821","nonce":2080816744}]},"public_addr":"172.21.15.192:6817/2080816744","cluster_addr":"172.21.15.192:6819/2080816744","heartbeat_back_addr":"172.21.15.192:6823/2080816744","heartbeat_front_addr":"172.21.15.192:6821/2080816744","state":["exists","up"]},{"osd":9,"uuid":"f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6827","nonce":1773691421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6829","nonce":1773691421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6833","nonce":1773691421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6831","nonce":1773691421}]},"public_addr":"172.21.15.122:6827/1773691421","cluster_addr":"172.21.15.122:6829/1773691421","heartbeat_back_addr":"172.21.15.122:6833/1773691421","heartbeat_front_addr":"172.21.15.122:6831/1773691421","state":["exists","up"]},{"osd":10,"uuid":"af88e50f-98f1-460d-856b-a920dd2ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6825","nonce":1954186137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6827","nonce":1954186137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6831","nonce":1954186137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6829","nonce":1954186137}]},"public_addr":"172.21.15.149:6825/1954186137","cluster_addr":"172.21.15.149:6827/1954186137","heartbeat_back_addr":"172.21.15.149:6831/1954186137","heartbeat_front_addr":"172.21.15.149:6829/1954186137","state":["exists","up"]},{"osd":11,"uuid":"b54c1b4d-cdb5-42d9-979a-47f9a870b5cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6825","nonce":2098474209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6827","nonce":2098474209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6831","nonce":2098474209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6829","nonce":2098474209}]},"public_addr":"172.21.15.192:6825/2098474209","cluster_addr":"172.21.15.192:6827/2098474209","heartbeat_back_addr":"172.21.15.192:6831/2098474209","heartbeat_front_addr":"172.21.15.192:6829/2098474209","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:46:50.049769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:13.518485+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:37.473023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:01.809640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:26.648514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:51.268056+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:16.294834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:42.079065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:07.609271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:33.885127+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:59.982708+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:51:26.461369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/2089907395":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6801/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:0/2001860625":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/1888284988":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/256574472":"2024-05-04T17:44:52.244049+0000","172.21.15.122:6801/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/64546794":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/584846864":"2024-05-04T17:45:18.454406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T17:53:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.248+0000 7fb652d8e700 1 -- 172.21.15.122:0/3467259492 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb63c0b2910 msgr2=0x7fb63c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.248+0000 7fb652d8e700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb63c0b2910 0x7fb63c0b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb63800b5c0 tx=0x7fb638005960 comp rx=0 tx=0).stop 2024-05-03T17:53:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.248+0000 7fb652d8e700 1 -- 172.21.15.122:0/3467259492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb64c1019f0 msgr2=0x7fb64c1a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.248+0000 7fb652d8e700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb64c1019f0 0x7fb64c1a04c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb64000b530 tx=0x7fb64000b840 comp rx=0 tx=0).stop 2024-05-03T17:53:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.248+0000 7fb652d8e700 1 -- 172.21.15.122:0/3467259492 shutdown_connections 2024-05-03T17:53:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.248+0000 7fb652d8e700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb63c0b2910 0x7fb63c0b4dd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.248+0000 7fb652d8e700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c108620 0x7fb64c1a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.248+0000 7fb652d8e700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb64c1019f0 0x7fb64c1a04c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.248+0000 7fb652d8e700 1 --2- 172.21.15.122:0/3467259492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb64c102490 0x7fb64c1a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.248+0000 7fb652d8e700 1 -- 172.21.15.122:0/3467259492 >> 172.21.15.122:0/3467259492 conn(0x7fb64c0789a0 msgr2=0x7fb64c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.249+0000 7fb652d8e700 1 -- 172.21.15.122:0/3467259492 shutdown_connections 2024-05-03T17:53:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:18.249+0000 7fb652d8e700 1 -- 172.21.15.122:0/3467259492 wait complete. 2024-05-03T17:53:18.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:17 smithi149 ceph-mon[33409]: pgmap v306: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-03T17:53:18.910 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-03T17:53:18.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T17:53:19.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:18 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3467259492' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:53:19.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-03T17:53:19.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-03T17:53:19.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-03T17:53:19.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-03T17:53:19.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-05-03T17:53:19.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:18 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-03T17:53:19.147 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:19.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:18 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3467259492' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:53:19.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-03T17:53:19.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-03T17:53:19.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-03T17:53:19.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-03T17:53:19.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-05-03T17:53:19.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:18 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-03T17:53:19.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:18 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3467259492' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:53:19.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-03T17:53:19.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-03T17:53:19.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-03T17:53:19.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-03T17:53:19.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-05-03T17:53:19.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:18 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-03T17:53:20.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:19 smithi122 ceph-mon[29744]: pgmap v307: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-03T17:53:20.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-03T17:53:20.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-03T17:53:20.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-03T17:53:20.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-03T17:53:20.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-05-03T17:53:20.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:19 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-03T17:53:20.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:19 smithi122 ceph-mon[29744]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T17:53:20.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:19 smithi149 ceph-mon[33409]: pgmap v307: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-03T17:53:20.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-03T17:53:20.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-03T17:53:20.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-03T17:53:20.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-03T17:53:20.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-05-03T17:53:20.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:19 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-03T17:53:20.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:19 smithi149 ceph-mon[33409]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T17:53:20.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:19 smithi192 ceph-mon[33753]: pgmap v307: 97 pgs: 97 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-03T17:53:20.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-03T17:53:20.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-03T17:53:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-03T17:53:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-03T17:53:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-05-03T17:53:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:19 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-03T17:53:20.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:19 smithi192 ceph-mon[33753]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T17:53:20.905 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:20 smithi122 ceph-mon[29744]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T17:53:20.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.913+0000 7f9a9ad62700 1 -- 172.21.15.122:0/659594311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a94108410 msgr2=0x7f9a941087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:20.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.913+0000 7f9a9ad62700 1 --2- 172.21.15.122:0/659594311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a94108410 0x7f9a941087f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009990 tx=0x7f9a88009ca0 comp rx=0 tx=0).stop 2024-05-03T17:53:20.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.913+0000 7f9a9ad62700 1 -- 172.21.15.122:0/659594311 shutdown_connections 2024-05-03T17:53:20.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.913+0000 7f9a9ad62700 1 --2- 172.21.15.122:0/659594311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a94102d60 0x7f9a9410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:20.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.913+0000 7f9a9ad62700 1 --2- 172.21.15.122:0/659594311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a94108410 0x7f9a941087f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:20.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.913+0000 7f9a9ad62700 1 --2- 172.21.15.122:0/659594311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a941023a0 0x7f9a94102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:20.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.913+0000 7f9a9ad62700 1 -- 172.21.15.122:0/659594311 >> 172.21.15.122:0/659594311 conn(0x7f9a94079350 msgr2=0x7f9a94079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:20.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a9ad62700 1 -- 172.21.15.122:0/659594311 shutdown_connections 2024-05-03T17:53:20.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a9ad62700 1 -- 172.21.15.122:0/659594311 wait complete. 2024-05-03T17:53:20.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a9ad62700 1 Processor -- start 2024-05-03T17:53:20.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a9ad62700 1 -- start start 2024-05-03T17:53:20.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a9ad62700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a941023a0 0x7f9a94104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a9ad62700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a94102d60 0x7f9a941045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a9ad62700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a94108410 0x7f9a94104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a9ad62700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9a94112530 con 0x7f9a94102d60 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a9ad62700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941123b0 con 0x7f9a94108410 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a9ad62700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941126b0 con 0x7f9a941023a0 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a93fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a94102d60 0x7f9a941045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a93fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a94102d60 0x7f9a941045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39228/0 (socket says 172.21.15.122:39228) 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.914+0000 7f9a93fff700 1 -- 172.21.15.122:0/2845657923 learned_addr learned my addr 172.21.15.122:0/2845657923 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a992ff700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a94108410 0x7f9a94104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a98afe700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a941023a0 0x7f9a94104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a93fff700 1 -- 172.21.15.122:0/2845657923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a941023a0 msgr2=0x7f9a94104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a93fff700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a941023a0 0x7f9a94104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:20.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a93fff700 1 -- 172.21.15.122:0/2845657923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a94108410 msgr2=0x7f9a94104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:20.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a93fff700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a94108410 0x7f9a94104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:20.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a93fff700 1 -- 172.21.15.122:0/2845657923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a941a80d0 con 0x7f9a94102d60 2024-05-03T17:53:20.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a98afe700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a941023a0 0x7f9a94104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:53:20.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a93fff700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a94102d60 0x7f9a941045c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000d950 tx=0x7f9a8000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:20.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a91ffb700 1 -- 172.21.15.122:0/2845657923 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80016070 con 0x7f9a94102d60 2024-05-03T17:53:20.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a91ffb700 1 -- 172.21.15.122:0/2845657923 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a80010460 con 0x7f9a94102d60 2024-05-03T17:53:20.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a9ad62700 1 -- 172.21.15.122:0/2845657923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a941a83c0 con 0x7f9a94102d60 2024-05-03T17:53:20.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a91ffb700 1 -- 172.21.15.122:0/2845657923 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8000f620 con 0x7f9a94102d60 2024-05-03T17:53:20.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.915+0000 7f9a9ad62700 1 -- 172.21.15.122:0/2845657923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a941a8950 con 0x7f9a94102d60 2024-05-03T17:53:20.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.916+0000 7f9a9ad62700 1 -- 172.21.15.122:0/2845657923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a94103ac0 con 0x7f9a94102d60 2024-05-03T17:53:20.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.917+0000 7f9a91ffb700 1 -- 172.21.15.122:0/2845657923 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f9a80019070 con 0x7f9a94102d60 2024-05-03T17:53:20.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.918+0000 7f9a91ffb700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a840b2990 0x7f9a840b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:20.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.918+0000 7f9a98afe700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a840b2990 0x7f9a840b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:20.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.919+0000 7f9a91ffb700 1 -- 172.21.15.122:0/2845657923 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f9a800e1f50 con 0x7f9a94102d60 2024-05-03T17:53:20.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.919+0000 7f9a98afe700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a840b2990 0x7f9a840b4e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800b5c0 tx=0x7f9a88005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:20.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:20.922+0000 7f9a91ffb700 1 -- 172.21.15.122:0/2845657923 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9a800a9b10 con 0x7f9a94102d60 2024-05-03T17:53:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.125+0000 7f9a9ad62700 1 -- 172.21.15.122:0/2845657923 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a941061c0 con 0x7f9a840b2990 2024-05-03T17:53:21.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.132+0000 7f9a91ffb700 1 -- 172.21.15.122:0/2845657923 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258869 (secure 0 0 0) 0x7f9a941061c0 con 0x7f9a840b2990 2024-05-03T17:53:21.131 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:53:21.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.136+0000 7f9a9ad62700 1 -- 172.21.15.122:0/2845657923 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a840b2990 msgr2=0x7f9a840b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:21.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.136+0000 7f9a9ad62700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a840b2990 0x7f9a840b4e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800b5c0 tx=0x7f9a88005fb0 comp rx=0 tx=0).stop 2024-05-03T17:53:21.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.136+0000 7f9a9ad62700 1 -- 172.21.15.122:0/2845657923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a94102d60 msgr2=0x7f9a941045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:21.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.136+0000 7f9a9ad62700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a94102d60 0x7f9a941045c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000d950 tx=0x7f9a8000dd10 comp rx=0 tx=0).stop 2024-05-03T17:53:21.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.136+0000 7f9a9ad62700 1 -- 172.21.15.122:0/2845657923 shutdown_connections 2024-05-03T17:53:21.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.136+0000 7f9a9ad62700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a840b2990 0x7f9a840b4e50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:21.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.136+0000 7f9a9ad62700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a94108410 0x7f9a94104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:21.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.136+0000 7f9a9ad62700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a941023a0 0x7f9a94104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:21.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.136+0000 7f9a9ad62700 1 --2- 172.21.15.122:0/2845657923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a94102d60 0x7f9a941045c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:21.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.136+0000 7f9a9ad62700 1 -- 172.21.15.122:0/2845657923 >> 172.21.15.122:0/2845657923 conn(0x7f9a94079350 msgr2=0x7f9a941016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:21.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.137+0000 7f9a9ad62700 1 -- 172.21.15.122:0/2845657923 shutdown_connections 2024-05-03T17:53:21.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:21.137+0000 7f9a9ad62700 1 -- 172.21.15.122:0/2845657923 wait complete. 2024-05-03T17:53:21.134 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-05-03T17:53:21.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:20 smithi149 ceph-mon[33409]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T17:53:21.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:20 smithi192 ceph-mon[33753]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T17:53:21.814 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-05-03T17:53:20.548283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644595,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":252,"num_read_kb":208,"num_write":282,"num_write_kb":2250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":290,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376044,"kb_used_data":10236,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565780,"statfs":{"total":1151940427776,"available":1151555358720,"internally_reserved":0,"allocated":10481664,"data_stored":6773533,"data_compressed":107130,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":45,"num_read_kb":20,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001831"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.870586+0000","last_change":"2024-05-03T17:52:35.725057+0000","last_active":"2024-05-03T17:53:18.870586+0000","last_peered":"2024-05-03T17:53:18.870586+0000","last_clean":"2024-05-03T17:53:18.870586+0000","last_became_active":"2024-05-03T17:52:35.724632+0000","last_became_peered":"2024-05-03T17:52:35.724632+0000","last_unstale":"2024-05-03T17:53:18.870586+0000","last_undegraded":"2024-05-03T17:53:18.870586+0000","last_fullsized":"2024-05-03T17:53:18.870586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:46.065357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722405+0000","last_change":"2024-05-03T17:52:35.728500+0000","last_active":"2024-05-03T17:52:37.722405+0000","last_peered":"2024-05-03T17:52:37.722405+0000","last_clean":"2024-05-03T17:52:37.722405+0000","last_became_active":"2024-05-03T17:52:35.728210+0000","last_became_peered":"2024-05-03T17:52:35.728210+0000","last_unstale":"2024-05-03T17:52:37.722405+0000","last_undegraded":"2024-05-03T17:52:37.722405+0000","last_fullsized":"2024-05-03T17:52:37.722405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:33:23.410617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722037+0000","last_change":"2024-05-03T17:52:35.726300+0000","last_active":"2024-05-03T17:52:37.722037+0000","last_peered":"2024-05-03T17:52:37.722037+0000","last_clean":"2024-05-03T17:52:37.722037+0000","last_became_active":"2024-05-03T17:52:35.725994+0000","last_became_peered":"2024-05-03T17:52:35.725994+0000","last_unstale":"2024-05-03T17:52:37.722037+0000","last_undegraded":"2024-05-03T17:52:37.722037+0000","last_fullsized":"2024-05-03T17:52:37.722037+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:31:19.723453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.867989+0000","last_change":"2024-05-03T17:52:35.728105+0000","last_active":"2024-05-03T17:53:18.867989+0000","last_peered":"2024-05-03T17:53:18.867989+0000","last_clean":"2024-05-03T17:53:18.867989+0000","last_became_active":"2024-05-03T17:52:35.727762+0000","last_became_peered":"2024-05-03T17:52:35.727762+0000","last_unstale":"2024-05-03T17:53:18.867989+0000","last_undegraded":"2024-05-03T17:53:18.867989+0000","last_fullsized":"2024-05-03T17:53:18.867989+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.640057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.872403+0000","last_change":"2024-05-03T17:52:35.734516+0000","last_active":"2024-05-03T17:53:18.872403+0000","last_peered":"2024-05-03T17:53:18.872403+0000","last_clean":"2024-05-03T17:53:18.872403+0000","last_became_active":"2024-05-03T17:52:35.734125+0000","last_became_peered":"2024-05-03T17:52:35.734125+0000","last_unstale":"2024-05-03T17:53:18.872403+0000","last_undegraded":"2024-05-03T17:53:18.872403+0000","last_fullsized":"2024-05-03T17:53:18.872403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:39:25.103137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.718777+0000","last_change":"2024-05-03T17:52:35.723427+0000","last_active":"2024-05-03T17:52:37.718777+0000","last_peered":"2024-05-03T17:52:37.718777+0000","last_clean":"2024-05-03T17:52:37.718777+0000","last_became_active":"2024-05-03T17:52:35.723113+0000","last_became_peered":"2024-05-03T17:52:35.723113+0000","last_unstale":"2024-05-03T17:52:37.718777+0000","last_undegraded":"2024-05-03T17:52:37.718777+0000","last_fullsized":"2024-05-03T17:52:37.718777+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:37:14.606347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.714785+0000","last_change":"2024-05-03T17:52:35.730858+0000","last_active":"2024-05-03T17:52:37.714785+0000","last_peered":"2024-05-03T17:52:37.714785+0000","last_clean":"2024-05-03T17:52:37.714785+0000","last_became_active":"2024-05-03T17:52:35.730453+0000","last_became_peered":"2024-05-03T17:52:35.730453+0000","last_unstale":"2024-05-03T17:52:37.714785+0000","last_undegraded":"2024-05-03T17:52:37.714785+0000","last_fullsized":"2024-05-03T17:52:37.714785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:21:17.803027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968183+0000","last_change":"2024-05-03T17:52:35.730547+0000","last_active":"2024-05-03T17:53:18.968183+0000","last_peered":"2024-05-03T17:53:18.968183+0000","last_clean":"2024-05-03T17:53:18.968183+0000","last_became_active":"2024-05-03T17:52:35.729582+0000","last_became_peered":"2024-05-03T17:52:35.729582+0000","last_unstale":"2024-05-03T17:53:18.968183+0000","last_undegraded":"2024-05-03T17:53:18.968183+0000","last_fullsized":"2024-05-03T17:53:18.968183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:24:55.110537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.724418+0000","last_change":"2024-05-03T17:52:35.726305+0000","last_active":"2024-05-03T17:52:37.724418+0000","last_peered":"2024-05-03T17:52:37.724418+0000","last_clean":"2024-05-03T17:52:37.724418+0000","last_became_active":"2024-05-03T17:52:35.725858+0000","last_became_peered":"2024-05-03T17:52:35.725858+0000","last_unstale":"2024-05-03T17:52:37.724418+0000","last_undegraded":"2024-05-03T17:52:37.724418+0000","last_fullsized":"2024-05-03T17:52:37.724418+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:59.347307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.719225+0000","last_change":"2024-05-03T17:52:35.726865+0000","last_active":"2024-05-03T17:52:37.719225+0000","last_peered":"2024-05-03T17:52:37.719225+0000","last_clean":"2024-05-03T17:52:37.719225+0000","last_became_active":"2024-05-03T17:52:35.726483+0000","last_became_peered":"2024-05-03T17:52:35.726483+0000","last_unstale":"2024-05-03T17:52:37.719225+0000","last_undegraded":"2024-05-03T17:52:37.719225+0000","last_fullsized":"2024-05-03T17:52:37.719225+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:17.851133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715343+0000","last_change":"2024-05-03T17:52:35.730551+0000","last_active":"2024-05-03T17:52:37.715343+0000","last_peered":"2024-05-03T17:52:37.715343+0000","last_clean":"2024-05-03T17:52:37.715343+0000","last_became_active":"2024-05-03T17:52:35.730187+0000","last_became_peered":"2024-05-03T17:52:35.730187+0000","last_unstale":"2024-05-03T17:52:37.715343+0000","last_undegraded":"2024-05-03T17:52:37.715343+0000","last_fullsized":"2024-05-03T17:52:37.715343+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:18:56.211061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.867924+0000","last_change":"2024-05-03T17:52:35.730040+0000","last_active":"2024-05-03T17:53:18.867924+0000","last_peered":"2024-05-03T17:53:18.867924+0000","last_clean":"2024-05-03T17:53:18.867924+0000","last_became_active":"2024-05-03T17:52:35.729549+0000","last_became_peered":"2024-05-03T17:52:35.729549+0000","last_unstale":"2024-05-03T17:53:18.867924+0000","last_undegraded":"2024-05-03T17:53:18.867924+0000","last_fullsized":"2024-05-03T17:53:18.867924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:50:05.052247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721722+0000","last_change":"2024-05-03T17:52:35.726709+0000","last_active":"2024-05-03T17:52:37.721722+0000","last_peered":"2024-05-03T17:52:37.721722+0000","last_clean":"2024-05-03T17:52:37.721722+0000","last_became_active":"2024-05-03T17:52:35.726166+0000","last_became_peered":"2024-05-03T17:52:35.726166+0000","last_unstale":"2024-05-03T17:52:37.721722+0000","last_undegraded":"2024-05-03T17:52:37.721722+0000","last_fullsized":"2024-05-03T17:52:37.721722+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.164622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722305+0000","last_change":"2024-05-03T17:52:35.727583+0000","last_active":"2024-05-03T17:52:37.722305+0000","last_peered":"2024-05-03T17:52:37.722305+0000","last_clean":"2024-05-03T17:52:37.722305+0000","last_became_active":"2024-05-03T17:52:35.727131+0000","last_became_peered":"2024-05-03T17:52:35.727131+0000","last_unstale":"2024-05-03T17:52:37.722305+0000","last_undegraded":"2024-05-03T17:52:37.722305+0000","last_fullsized":"2024-05-03T17:52:37.722305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:28:21.255954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721652+0000","last_change":"2024-05-03T17:52:35.726253+0000","last_active":"2024-05-03T17:52:37.721652+0000","last_peered":"2024-05-03T17:52:37.721652+0000","last_clean":"2024-05-03T17:52:37.721652+0000","last_became_active":"2024-05-03T17:52:35.725491+0000","last_became_peered":"2024-05-03T17:52:35.725491+0000","last_unstale":"2024-05-03T17:52:37.721652+0000","last_undegraded":"2024-05-03T17:52:37.721652+0000","last_fullsized":"2024-05-03T17:52:37.721652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:52:28.424299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721821+0000","last_change":"2024-05-03T17:52:35.717354+0000","last_active":"2024-05-03T17:52:37.721821+0000","last_peered":"2024-05-03T17:52:37.721821+0000","last_clean":"2024-05-03T17:52:37.721821+0000","last_became_active":"2024-05-03T17:52:35.717025+0000","last_became_peered":"2024-05-03T17:52:35.717025+0000","last_unstale":"2024-05-03T17:52:37.721821+0000","last_undegraded":"2024-05-03T17:52:37.721821+0000","last_fullsized":"2024-05-03T17:52:37.721821+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:11:07.778484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.870767+0000","last_change":"2024-05-03T17:52:35.725626+0000","last_active":"2024-05-03T17:53:18.870767+0000","last_peered":"2024-05-03T17:53:18.870767+0000","last_clean":"2024-05-03T17:53:18.870767+0000","last_became_active":"2024-05-03T17:52:35.725239+0000","last_became_peered":"2024-05-03T17:52:35.725239+0000","last_unstale":"2024-05-03T17:53:18.870767+0000","last_undegraded":"2024-05-03T17:53:18.870767+0000","last_fullsized":"2024-05-03T17:53:18.870767+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:44:18.359484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721156+0000","last_change":"2024-05-03T17:52:35.724250+0000","last_active":"2024-05-03T17:52:37.721156+0000","last_peered":"2024-05-03T17:52:37.721156+0000","last_clean":"2024-05-03T17:52:37.721156+0000","last_became_active":"2024-05-03T17:52:35.723998+0000","last_became_peered":"2024-05-03T17:52:35.723998+0000","last_unstale":"2024-05-03T17:52:37.721156+0000","last_undegraded":"2024-05-03T17:52:37.721156+0000","last_fullsized":"2024-05-03T17:52:37.721156+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:03:32.939559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886262+0000","last_change":"2024-05-03T17:52:35.726237+0000","last_active":"2024-05-03T17:53:19.886262+0000","last_peered":"2024-05-03T17:53:19.886262+0000","last_clean":"2024-05-03T17:53:19.886262+0000","last_became_active":"2024-05-03T17:52:35.725847+0000","last_became_peered":"2024-05-03T17:52:35.725847+0000","last_unstale":"2024-05-03T17:53:19.886262+0000","last_undegraded":"2024-05-03T17:53:19.886262+0000","last_fullsized":"2024-05-03T17:53:19.886262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:06.472532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:50.790568+0000","last_change":"2024-05-03T17:52:32.687198+0000","last_active":"2024-05-03T17:52:50.790568+0000","last_peered":"2024-05-03T17:52:50.790568+0000","last_clean":"2024-05-03T17:52:50.790568+0000","last_became_active":"2024-05-03T17:52:32.686848+0000","last_became_peered":"2024-05-03T17:52:32.686848+0000","last_unstale":"2024-05-03T17:52:50.790568+0000","last_undegraded":"2024-05-03T17:52:50.790568+0000","last_fullsized":"2024-05-03T17:52:50.790568+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:08:24.244371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968473+0000","last_change":"2024-05-03T17:52:35.730595+0000","last_active":"2024-05-03T17:53:18.968473+0000","last_peered":"2024-05-03T17:53:18.968473+0000","last_clean":"2024-05-03T17:53:18.968473+0000","last_became_active":"2024-05-03T17:52:35.729999+0000","last_became_peered":"2024-05-03T17:52:35.729999+0000","last_unstale":"2024-05-03T17:53:18.968473+0000","last_undegraded":"2024-05-03T17:53:18.968473+0000","last_fullsized":"2024-05-03T17:53:18.968473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:14.485756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968634+0000","last_change":"2024-05-03T17:52:32.935239+0000","last_active":"2024-05-03T17:53:18.968634+0000","last_peered":"2024-05-03T17:53:18.968634+0000","last_clean":"2024-05-03T17:53:18.968634+0000","last_became_active":"2024-05-03T17:52:32.934915+0000","last_became_peered":"2024-05-03T17:52:32.934915+0000","last_unstale":"2024-05-03T17:53:18.968634+0000","last_undegraded":"2024-05-03T17:53:18.968634+0000","last_fullsized":"2024-05-03T17:53:18.968634+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:20.785071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.723294+0000","last_change":"2024-05-03T17:52:35.724940+0000","last_active":"2024-05-03T17:52:37.723294+0000","last_peered":"2024-05-03T17:52:37.723294+0000","last_clean":"2024-05-03T17:52:37.723294+0000","last_became_active":"2024-05-03T17:52:35.724603+0000","last_became_peered":"2024-05-03T17:52:35.724603+0000","last_unstale":"2024-05-03T17:52:37.723294+0000","last_undegraded":"2024-05-03T17:52:37.723294+0000","last_fullsized":"2024-05-03T17:52:37.723294+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:43.640499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873517+0000","last_change":"2024-05-03T17:52:32.688202+0000","last_active":"2024-05-03T17:53:18.873517+0000","last_peered":"2024-05-03T17:53:18.873517+0000","last_clean":"2024-05-03T17:53:18.873517+0000","last_became_active":"2024-05-03T17:52:32.687893+0000","last_became_peered":"2024-05-03T17:52:32.687893+0000","last_unstale":"2024-05-03T17:53:18.873517+0000","last_undegraded":"2024-05-03T17:53:18.873517+0000","last_fullsized":"2024-05-03T17:53:18.873517+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:43.342832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722067+0000","last_change":"2024-05-03T17:52:35.720980+0000","last_active":"2024-05-03T17:52:37.722067+0000","last_peered":"2024-05-03T17:52:37.722067+0000","last_clean":"2024-05-03T17:52:37.722067+0000","last_became_active":"2024-05-03T17:52:35.720582+0000","last_became_peered":"2024-05-03T17:52:35.720582+0000","last_unstale":"2024-05-03T17:52:37.722067+0000","last_undegraded":"2024-05-03T17:52:37.722067+0000","last_fullsized":"2024-05-03T17:52:37.722067+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:01:22.938186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":2,"reported_epoch":78,"state":"peering","last_fresh":"2024-05-03T17:52:37.715245+0000","last_change":"2024-05-03T17:53:19.412769+0000","last_active":"2024-05-03T17:53:19.412769+0000","last_peered":"2024-05-03T17:52:37.715245+0000","last_clean":"2024-05-03T17:52:37.715245+0000","last_became_active":"2024-05-03T17:52:32.688073+0000","last_became_peered":"2024-05-03T17:52:32.688073+0000","last_unstale":"2024-05-03T17:52:37.715245+0000","last_undegraded":"2024-05-03T17:52:37.715245+0000","last_fullsized":"2024-05-03T17:52:37.715245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.723561+0000","last_change":"2024-05-03T17:52:35.730268+0000","last_active":"2024-05-03T17:52:37.723561+0000","last_peered":"2024-05-03T17:52:37.723561+0000","last_clean":"2024-05-03T17:52:37.723561+0000","last_became_active":"2024-05-03T17:52:35.729315+0000","last_became_peered":"2024-05-03T17:52:35.729315+0000","last_unstale":"2024-05-03T17:52:37.723561+0000","last_undegraded":"2024-05-03T17:52:37.723561+0000","last_fullsized":"2024-05-03T17:52:37.723561+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:25.584453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873688+0000","last_change":"2024-05-03T17:52:32.692735+0000","last_active":"2024-05-03T17:53:18.873688+0000","last_peered":"2024-05-03T17:53:18.873688+0000","last_clean":"2024-05-03T17:53:18.873688+0000","last_became_active":"2024-05-03T17:52:32.692394+0000","last_became_peered":"2024-05-03T17:52:32.692394+0000","last_unstale":"2024-05-03T17:53:18.873688+0000","last_undegraded":"2024-05-03T17:53:18.873688+0000","last_fullsized":"2024-05-03T17:53:18.873688+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:40:11.434751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.872637+0000","last_change":"2024-05-03T17:52:35.727167+0000","last_active":"2024-05-03T17:53:18.872637+0000","last_peered":"2024-05-03T17:53:18.872637+0000","last_clean":"2024-05-03T17:53:18.872637+0000","last_became_active":"2024-05-03T17:52:35.726792+0000","last_became_peered":"2024-05-03T17:52:35.726792+0000","last_unstale":"2024-05-03T17:53:18.872637+0000","last_undegraded":"2024-05-03T17:53:18.872637+0000","last_fullsized":"2024-05-03T17:53:18.872637+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:54:01.823025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:50.788699+0000","last_change":"2024-05-03T17:52:32.687573+0000","last_active":"2024-05-03T17:52:50.788699+0000","last_peered":"2024-05-03T17:52:50.788699+0000","last_clean":"2024-05-03T17:52:50.788699+0000","last_became_active":"2024-05-03T17:52:32.686158+0000","last_became_peered":"2024-05-03T17:52:32.686158+0000","last_unstale":"2024-05-03T17:52:50.788699+0000","last_undegraded":"2024-05-03T17:52:50.788699+0000","last_fullsized":"2024-05-03T17:52:50.788699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:19.891479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873865+0000","last_change":"2024-05-03T17:52:35.728049+0000","last_active":"2024-05-03T17:53:18.873865+0000","last_peered":"2024-05-03T17:53:18.873865+0000","last_clean":"2024-05-03T17:53:18.873865+0000","last_became_active":"2024-05-03T17:52:35.727713+0000","last_became_peered":"2024-05-03T17:52:35.727713+0000","last_unstale":"2024-05-03T17:53:18.873865+0000","last_undegraded":"2024-05-03T17:53:18.873865+0000","last_fullsized":"2024-05-03T17:53:18.873865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:28:01.798457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:50.788759+0000","last_change":"2024-05-03T17:52:32.685679+0000","last_active":"2024-05-03T17:52:50.788759+0000","last_peered":"2024-05-03T17:52:50.788759+0000","last_clean":"2024-05-03T17:52:50.788759+0000","last_became_active":"2024-05-03T17:52:32.685306+0000","last_became_peered":"2024-05-03T17:52:32.685306+0000","last_unstale":"2024-05-03T17:52:50.788759+0000","last_undegraded":"2024-05-03T17:52:50.788759+0000","last_fullsized":"2024-05-03T17:52:50.788759+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:48:22.218310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968144+0000","last_change":"2024-05-03T17:52:35.717111+0000","last_active":"2024-05-03T17:53:18.968144+0000","last_peered":"2024-05-03T17:53:18.968144+0000","last_clean":"2024-05-03T17:53:18.968144+0000","last_became_active":"2024-05-03T17:52:35.716782+0000","last_became_peered":"2024-05-03T17:52:35.716782+0000","last_unstale":"2024-05-03T17:53:18.968144+0000","last_undegraded":"2024-05-03T17:53:18.968144+0000","last_fullsized":"2024-05-03T17:53:18.968144+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:31:57.766927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873103+0000","last_change":"2024-05-03T17:52:32.690846+0000","last_active":"2024-05-03T17:53:18.873103+0000","last_peered":"2024-05-03T17:53:18.873103+0000","last_clean":"2024-05-03T17:53:18.873103+0000","last_became_active":"2024-05-03T17:52:32.690506+0000","last_became_peered":"2024-05-03T17:52:32.690506+0000","last_unstale":"2024-05-03T17:53:18.873103+0000","last_undegraded":"2024-05-03T17:53:18.873103+0000","last_fullsized":"2024-05-03T17:53:18.873103+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:19.810205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.720459+0000","last_change":"2024-05-03T17:52:35.718229+0000","last_active":"2024-05-03T17:52:37.720459+0000","last_peered":"2024-05-03T17:52:37.720459+0000","last_clean":"2024-05-03T17:52:37.720459+0000","last_became_active":"2024-05-03T17:52:35.717907+0000","last_became_peered":"2024-05-03T17:52:35.717907+0000","last_unstale":"2024-05-03T17:52:37.720459+0000","last_undegraded":"2024-05-03T17:52:37.720459+0000","last_fullsized":"2024-05-03T17:52:37.720459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:45.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":49,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:53:15.798602+0000","last_change":"2024-05-03T17:52:32.691174+0000","last_active":"2024-05-03T17:53:15.798602+0000","last_peered":"2024-05-03T17:53:15.798602+0000","last_clean":"2024-05-03T17:53:15.798602+0000","last_became_active":"2024-05-03T17:52:32.689907+0000","last_became_peered":"2024-05-03T17:52:32.689907+0000","last_unstale":"2024-05-03T17:53:15.798602+0000","last_undegraded":"2024-05-03T17:53:15.798602+0000","last_fullsized":"2024-05-03T17:53:15.798602+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:46.630436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.724070+0000","last_change":"2024-05-03T17:52:35.720290+0000","last_active":"2024-05-03T17:52:37.724070+0000","last_peered":"2024-05-03T17:52:37.724070+0000","last_clean":"2024-05-03T17:52:37.724070+0000","last_became_active":"2024-05-03T17:52:35.719962+0000","last_became_peered":"2024-05-03T17:52:35.719962+0000","last_unstale":"2024-05-03T17:52:37.724070+0000","last_undegraded":"2024-05-03T17:52:37.724070+0000","last_fullsized":"2024-05-03T17:52:37.724070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:31:41.301592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:49.781041+0000","last_change":"2024-05-03T17:52:32.689937+0000","last_active":"2024-05-03T17:52:49.781041+0000","last_peered":"2024-05-03T17:52:49.781041+0000","last_clean":"2024-05-03T17:52:49.781041+0000","last_became_active":"2024-05-03T17:52:32.689624+0000","last_became_peered":"2024-05-03T17:52:32.689624+0000","last_unstale":"2024-05-03T17:52:49.781041+0000","last_undegraded":"2024-05-03T17:52:49.781041+0000","last_fullsized":"2024-05-03T17:52:49.781041+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:24.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.723130+0000","last_change":"2024-05-03T17:52:35.726824+0000","last_active":"2024-05-03T17:52:37.723130+0000","last_peered":"2024-05-03T17:52:37.723130+0000","last_clean":"2024-05-03T17:52:37.723130+0000","last_became_active":"2024-05-03T17:52:35.726494+0000","last_became_peered":"2024-05-03T17:52:35.726494+0000","last_unstale":"2024-05-03T17:52:37.723130+0000","last_undegraded":"2024-05-03T17:52:37.723130+0000","last_fullsized":"2024-05-03T17:52:37.723130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:41:21.284852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873930+0000","last_change":"2024-05-03T17:52:32.935471+0000","last_active":"2024-05-03T17:53:18.873930+0000","last_peered":"2024-05-03T17:53:18.873930+0000","last_clean":"2024-05-03T17:53:18.873930+0000","last_became_active":"2024-05-03T17:52:32.935012+0000","last_became_peered":"2024-05-03T17:52:32.935012+0000","last_unstale":"2024-05-03T17:53:18.873930+0000","last_undegraded":"2024-05-03T17:53:18.873930+0000","last_fullsized":"2024-05-03T17:53:18.873930+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:35:38.654577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.723823+0000","last_change":"2024-05-03T17:52:35.727124+0000","last_active":"2024-05-03T17:52:37.723823+0000","last_peered":"2024-05-03T17:52:37.723823+0000","last_clean":"2024-05-03T17:52:37.723823+0000","last_became_active":"2024-05-03T17:52:35.726804+0000","last_became_peered":"2024-05-03T17:52:35.726804+0000","last_unstale":"2024-05-03T17:52:37.723823+0000","last_undegraded":"2024-05-03T17:52:37.723823+0000","last_fullsized":"2024-05-03T17:52:37.723823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:32.000803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'25","reported_seq":102,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:53:16.799915+0000","last_change":"2024-05-03T17:52:32.685014+0000","last_active":"2024-05-03T17:53:16.799915+0000","last_peered":"2024-05-03T17:53:16.799915+0000","last_clean":"2024-05-03T17:53:16.799915+0000","last_became_active":"2024-05-03T17:52:32.684660+0000","last_became_peered":"2024-05-03T17:52:32.684660+0000","last_unstale":"2024-05-03T17:53:16.799915+0000","last_undegraded":"2024-05-03T17:53:16.799915+0000","last_fullsized":"2024-05-03T17:53:16.799915+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:25:56.890283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49601,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":55,"num_read_kb":75,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873225+0000","last_change":"2024-05-03T17:52:35.733675+0000","last_active":"2024-05-03T17:53:18.873225+0000","last_peered":"2024-05-03T17:53:18.873225+0000","last_clean":"2024-05-03T17:53:18.873225+0000","last_became_active":"2024-05-03T17:52:35.733234+0000","last_became_peered":"2024-05-03T17:52:35.733234+0000","last_unstale":"2024-05-03T17:53:18.873225+0000","last_undegraded":"2024-05-03T17:53:18.873225+0000","last_fullsized":"2024-05-03T17:53:18.873225+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:57.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886182+0000","last_change":"2024-05-03T17:52:32.690462+0000","last_active":"2024-05-03T17:53:19.886182+0000","last_peered":"2024-05-03T17:53:19.886182+0000","last_clean":"2024-05-03T17:53:19.886182+0000","last_became_active":"2024-05-03T17:52:32.690169+0000","last_became_peered":"2024-05-03T17:52:32.690169+0000","last_unstale":"2024-05-03T17:53:19.886182+0000","last_undegraded":"2024-05-03T17:53:19.886182+0000","last_fullsized":"2024-05-03T17:53:19.886182+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:59.324612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721144+0000","last_change":"2024-05-03T17:52:35.720730+0000","last_active":"2024-05-03T17:52:37.721144+0000","last_peered":"2024-05-03T17:52:37.721144+0000","last_clean":"2024-05-03T17:52:37.721144+0000","last_became_active":"2024-05-03T17:52:35.720392+0000","last_became_peered":"2024-05-03T17:52:35.720392+0000","last_unstale":"2024-05-03T17:52:37.721144+0000","last_undegraded":"2024-05-03T17:52:37.721144+0000","last_fullsized":"2024-05-03T17:52:37.721144+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:58.528148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873704+0000","last_change":"2024-05-03T17:52:32.691439+0000","last_active":"2024-05-03T17:53:18.873704+0000","last_peered":"2024-05-03T17:53:18.873704+0000","last_clean":"2024-05-03T17:53:18.873704+0000","last_became_active":"2024-05-03T17:52:32.691080+0000","last_became_peered":"2024-05-03T17:52:32.691080+0000","last_unstale":"2024-05-03T17:53:18.873704+0000","last_undegraded":"2024-05-03T17:53:18.873704+0000","last_fullsized":"2024-05-03T17:53:18.873704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:40:43.120194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715331+0000","last_change":"2024-05-03T17:52:35.729860+0000","last_active":"2024-05-03T17:52:37.715331+0000","last_peered":"2024-05-03T17:52:37.715331+0000","last_clean":"2024-05-03T17:52:37.715331+0000","last_became_active":"2024-05-03T17:52:35.729508+0000","last_became_peered":"2024-05-03T17:52:35.729508+0000","last_unstale":"2024-05-03T17:52:37.715331+0000","last_undegraded":"2024-05-03T17:52:37.715331+0000","last_fullsized":"2024-05-03T17:52:37.715331+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:39:20.859215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.868360+0000","last_change":"2024-05-03T17:52:32.689346+0000","last_active":"2024-05-03T17:53:18.868360+0000","last_peered":"2024-05-03T17:53:18.868360+0000","last_clean":"2024-05-03T17:53:18.868360+0000","last_became_active":"2024-05-03T17:52:32.689009+0000","last_became_peered":"2024-05-03T17:52:32.689009+0000","last_unstale":"2024-05-03T17:53:18.868360+0000","last_undegraded":"2024-05-03T17:53:18.868360+0000","last_fullsized":"2024-05-03T17:53:18.868360+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:55:36.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.714758+0000","last_change":"2024-05-03T17:52:35.729226+0000","last_active":"2024-05-03T17:52:37.714758+0000","last_peered":"2024-05-03T17:52:37.714758+0000","last_clean":"2024-05-03T17:52:37.714758+0000","last_became_active":"2024-05-03T17:52:35.728846+0000","last_became_peered":"2024-05-03T17:52:35.728846+0000","last_unstale":"2024-05-03T17:52:37.714758+0000","last_undegraded":"2024-05-03T17:52:37.714758+0000","last_fullsized":"2024-05-03T17:52:37.714758+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:39:16.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":39,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.872871+0000","last_change":"2024-05-03T17:52:32.936108+0000","last_active":"2024-05-03T17:53:18.872871+0000","last_peered":"2024-05-03T17:53:18.872871+0000","last_clean":"2024-05-03T17:53:18.872871+0000","last_became_active":"2024-05-03T17:52:32.935782+0000","last_became_peered":"2024-05-03T17:52:32.935782+0000","last_unstale":"2024-05-03T17:53:18.872871+0000","last_undegraded":"2024-05-03T17:53:18.872871+0000","last_fullsized":"2024-05-03T17:53:18.872871+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:47.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.874458+0000","last_change":"2024-05-03T17:52:35.727163+0000","last_active":"2024-05-03T17:53:18.874458+0000","last_peered":"2024-05-03T17:53:18.874458+0000","last_clean":"2024-05-03T17:53:18.874458+0000","last_became_active":"2024-05-03T17:52:35.726824+0000","last_became_peered":"2024-05-03T17:52:35.726824+0000","last_unstale":"2024-05-03T17:53:18.874458+0000","last_undegraded":"2024-05-03T17:53:18.874458+0000","last_fullsized":"2024-05-03T17:53:18.874458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:24:50.803860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873246+0000","last_change":"2024-05-03T17:52:32.687329+0000","last_active":"2024-05-03T17:53:18.873246+0000","last_peered":"2024-05-03T17:53:18.873246+0000","last_clean":"2024-05-03T17:53:18.873246+0000","last_became_active":"2024-05-03T17:52:32.687025+0000","last_became_peered":"2024-05-03T17:52:32.687025+0000","last_unstale":"2024-05-03T17:53:18.873246+0000","last_undegraded":"2024-05-03T17:53:18.873246+0000","last_fullsized":"2024-05-03T17:53:18.873246+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:42:15.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.870328+0000","last_change":"2024-05-03T17:52:35.728797+0000","last_active":"2024-05-03T17:53:18.870328+0000","last_peered":"2024-05-03T17:53:18.870328+0000","last_clean":"2024-05-03T17:53:18.870328+0000","last_became_active":"2024-05-03T17:52:35.728429+0000","last_became_peered":"2024-05-03T17:52:35.728429+0000","last_unstale":"2024-05-03T17:53:18.870328+0000","last_undegraded":"2024-05-03T17:53:18.870328+0000","last_fullsized":"2024-05-03T17:53:18.870328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:19.584593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.720872+0000","last_change":"2024-05-03T17:52:32.935617+0000","last_active":"2024-05-03T17:52:37.720872+0000","last_peered":"2024-05-03T17:52:37.720872+0000","last_clean":"2024-05-03T17:52:37.720872+0000","last_became_active":"2024-05-03T17:52:32.935271+0000","last_became_peered":"2024-05-03T17:52:32.935271+0000","last_unstale":"2024-05-03T17:52:37.720872+0000","last_undegraded":"2024-05-03T17:52:37.720872+0000","last_fullsized":"2024-05-03T17:52:37.720872+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:53:28.540396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.719071+0000","last_change":"2024-05-03T17:52:35.719115+0000","last_active":"2024-05-03T17:52:37.719071+0000","last_peered":"2024-05-03T17:52:37.719071+0000","last_clean":"2024-05-03T17:52:37.719071+0000","last_became_active":"2024-05-03T17:52:35.718054+0000","last_became_peered":"2024-05-03T17:52:35.718054+0000","last_unstale":"2024-05-03T17:52:37.719071+0000","last_undegraded":"2024-05-03T17:52:37.719071+0000","last_fullsized":"2024-05-03T17:52:37.719071+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.903766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715066+0000","last_change":"2024-05-03T17:52:32.688993+0000","last_active":"2024-05-03T17:52:37.715066+0000","last_peered":"2024-05-03T17:52:37.715066+0000","last_clean":"2024-05-03T17:52:37.715066+0000","last_became_active":"2024-05-03T17:52:32.688706+0000","last_became_peered":"2024-05-03T17:52:32.688706+0000","last_unstale":"2024-05-03T17:52:37.715066+0000","last_undegraded":"2024-05-03T17:52:37.715066+0000","last_fullsized":"2024-05-03T17:52:37.715066+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:40:14.023338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.718870+0000","last_change":"2024-05-03T17:52:35.724023+0000","last_active":"2024-05-03T17:52:37.718870+0000","last_peered":"2024-05-03T17:52:37.718870+0000","last_clean":"2024-05-03T17:52:37.718870+0000","last_became_active":"2024-05-03T17:52:35.723696+0000","last_became_peered":"2024-05-03T17:52:35.723696+0000","last_unstale":"2024-05-03T17:52:37.718870+0000","last_undegraded":"2024-05-03T17:52:37.718870+0000","last_fullsized":"2024-05-03T17:52:37.718870+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:06:36.805748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873912+0000","last_change":"2024-05-03T17:52:35.728958+0000","last_active":"2024-05-03T17:53:18.873912+0000","last_peered":"2024-05-03T17:53:18.873912+0000","last_clean":"2024-05-03T17:53:18.873912+0000","last_became_active":"2024-05-03T17:52:35.728619+0000","last_became_peered":"2024-05-03T17:52:35.728619+0000","last_unstale":"2024-05-03T17:53:18.873912+0000","last_undegraded":"2024-05-03T17:53:18.873912+0000","last_fullsized":"2024-05-03T17:53:18.873912+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:29:29.922324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:53:16.800156+0000","last_change":"2024-05-03T17:52:32.935367+0000","last_active":"2024-05-03T17:53:16.800156+0000","last_peered":"2024-05-03T17:53:16.800156+0000","last_clean":"2024-05-03T17:53:16.800156+0000","last_became_active":"2024-05-03T17:52:32.934938+0000","last_became_peered":"2024-05-03T17:52:32.934938+0000","last_unstale":"2024-05-03T17:53:16.800156+0000","last_undegraded":"2024-05-03T17:53:16.800156+0000","last_fullsized":"2024-05-03T17:53:16.800156+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:07:21.642883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968345+0000","last_change":"2024-05-03T17:52:35.731035+0000","last_active":"2024-05-03T17:53:18.968345+0000","last_peered":"2024-05-03T17:53:18.968345+0000","last_clean":"2024-05-03T17:53:18.968345+0000","last_became_active":"2024-05-03T17:52:35.730034+0000","last_became_peered":"2024-05-03T17:52:35.730034+0000","last_unstale":"2024-05-03T17:53:18.968345+0000","last_undegraded":"2024-05-03T17:53:18.968345+0000","last_fullsized":"2024-05-03T17:53:18.968345+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:25.545290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.867900+0000","last_change":"2024-05-03T17:52:32.690043+0000","last_active":"2024-05-03T17:53:18.867900+0000","last_peered":"2024-05-03T17:53:18.867900+0000","last_clean":"2024-05-03T17:53:18.867900+0000","last_became_active":"2024-05-03T17:52:32.689562+0000","last_became_peered":"2024-05-03T17:52:32.689562+0000","last_unstale":"2024-05-03T17:53:18.867900+0000","last_undegraded":"2024-05-03T17:53:18.867900+0000","last_fullsized":"2024-05-03T17:53:18.867900+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:59.014968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":93,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.714703+0000","last_change":"2024-05-03T17:51:02.340854+0000","last_active":"2024-05-03T17:52:37.714703+0000","last_peered":"2024-05-03T17:52:37.714703+0000","last_clean":"2024-05-03T17:52:37.714703+0000","last_became_active":"2024-05-03T17:51:02.336454+0000","last_became_peered":"2024-05-03T17:51:02.336454+0000","last_unstale":"2024-05-03T17:52:37.714703+0000","last_undegraded":"2024-05-03T17:52:37.714703+0000","last_fullsized":"2024-05-03T17:52:37.714703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_clean_scrub_stamp":"2024-05-03T17:47:41.306771+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:18:08.512098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.720611+0000","last_change":"2024-05-03T17:52:35.727303+0000","last_active":"2024-05-03T17:52:37.720611+0000","last_peered":"2024-05-03T17:52:37.720611+0000","last_clean":"2024-05-03T17:52:37.720611+0000","last_became_active":"2024-05-03T17:52:35.726951+0000","last_became_peered":"2024-05-03T17:52:35.726951+0000","last_unstale":"2024-05-03T17:52:37.720611+0000","last_undegraded":"2024-05-03T17:52:37.720611+0000","last_fullsized":"2024-05-03T17:52:37.720611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:20:31.650582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:50.789304+0000","last_change":"2024-05-03T17:52:32.690093+0000","last_active":"2024-05-03T17:52:50.789304+0000","last_peered":"2024-05-03T17:52:50.789304+0000","last_clean":"2024-05-03T17:52:50.789304+0000","last_became_active":"2024-05-03T17:52:32.689777+0000","last_became_peered":"2024-05-03T17:52:32.689777+0000","last_unstale":"2024-05-03T17:52:50.789304+0000","last_undegraded":"2024-05-03T17:52:50.789304+0000","last_fullsized":"2024-05-03T17:52:50.789304+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:09.693011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873331+0000","last_change":"2024-05-03T17:52:32.935589+0000","last_active":"2024-05-03T17:53:18.873331+0000","last_peered":"2024-05-03T17:53:18.873331+0000","last_clean":"2024-05-03T17:53:18.873331+0000","last_became_active":"2024-05-03T17:52:32.935221+0000","last_became_peered":"2024-05-03T17:52:32.935221+0000","last_unstale":"2024-05-03T17:53:18.873331+0000","last_undegraded":"2024-05-03T17:53:18.873331+0000","last_fullsized":"2024-05-03T17:53:18.873331+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:26:08.001761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721113+0000","last_change":"2024-05-03T17:52:35.724293+0000","last_active":"2024-05-03T17:52:37.721113+0000","last_peered":"2024-05-03T17:52:37.721113+0000","last_clean":"2024-05-03T17:52:37.721113+0000","last_became_active":"2024-05-03T17:52:35.723978+0000","last_became_peered":"2024-05-03T17:52:35.723978+0000","last_unstale":"2024-05-03T17:52:37.721113+0000","last_undegraded":"2024-05-03T17:52:37.721113+0000","last_fullsized":"2024-05-03T17:52:37.721113+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:23.357872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'9","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:53:10.724007+0000","last_change":"2024-05-03T17:52:32.687303+0000","last_active":"2024-05-03T17:53:10.724007+0000","last_peered":"2024-05-03T17:53:10.724007+0000","last_clean":"2024-05-03T17:53:10.724007+0000","last_became_active":"2024-05-03T17:52:32.686827+0000","last_became_peered":"2024-05-03T17:52:32.686827+0000","last_unstale":"2024-05-03T17:53:10.724007+0000","last_undegraded":"2024-05-03T17:53:10.724007+0000","last_fullsized":"2024-05-03T17:53:10.724007+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:11:53.565215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968621+0000","last_change":"2024-05-03T17:52:35.720898+0000","last_active":"2024-05-03T17:53:18.968621+0000","last_peered":"2024-05-03T17:53:18.968621+0000","last_clean":"2024-05-03T17:53:18.968621+0000","last_became_active":"2024-05-03T17:52:35.720572+0000","last_became_peered":"2024-05-03T17:52:35.720572+0000","last_unstale":"2024-05-03T17:53:18.968621+0000","last_undegraded":"2024-05-03T17:53:18.968621+0000","last_fullsized":"2024-05-03T17:53:18.968621+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:01.697167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.868053+0000","last_change":"2024-05-03T17:52:33.249970+0000","last_active":"2024-05-03T17:53:18.868053+0000","last_peered":"2024-05-03T17:53:18.868053+0000","last_clean":"2024-05-03T17:53:18.868053+0000","last_became_active":"2024-05-03T17:52:33.249596+0000","last_became_peered":"2024-05-03T17:52:33.249596+0000","last_unstale":"2024-05-03T17:53:18.868053+0000","last_undegraded":"2024-05-03T17:53:18.868053+0000","last_fullsized":"2024-05-03T17:53:18.868053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:46:53.058586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722553+0000","last_change":"2024-05-03T17:52:35.722129+0000","last_active":"2024-05-03T17:52:37.722553+0000","last_peered":"2024-05-03T17:52:37.722553+0000","last_clean":"2024-05-03T17:52:37.722553+0000","last_became_active":"2024-05-03T17:52:35.721815+0000","last_became_peered":"2024-05-03T17:52:35.721815+0000","last_unstale":"2024-05-03T17:52:37.722553+0000","last_undegraded":"2024-05-03T17:52:37.722553+0000","last_fullsized":"2024-05-03T17:52:37.722553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:33.781350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:49.781271+0000","last_change":"2024-05-03T17:52:32.936157+0000","last_active":"2024-05-03T17:52:49.781271+0000","last_peered":"2024-05-03T17:52:49.781271+0000","last_clean":"2024-05-03T17:52:49.781271+0000","last_became_active":"2024-05-03T17:52:32.935842+0000","last_became_peered":"2024-05-03T17:52:32.935842+0000","last_unstale":"2024-05-03T17:52:49.781271+0000","last_undegraded":"2024-05-03T17:52:49.781271+0000","last_fullsized":"2024-05-03T17:52:49.781271+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:18.006451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.872881+0000","last_change":"2024-05-03T17:52:35.735993+0000","last_active":"2024-05-03T17:53:18.872881+0000","last_peered":"2024-05-03T17:53:18.872881+0000","last_clean":"2024-05-03T17:53:18.872881+0000","last_became_active":"2024-05-03T17:52:35.735612+0000","last_became_peered":"2024-05-03T17:52:35.735612+0000","last_unstale":"2024-05-03T17:53:18.872881+0000","last_undegraded":"2024-05-03T17:53:18.872881+0000","last_fullsized":"2024-05-03T17:53:18.872881+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:59.216022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968246+0000","last_change":"2024-05-03T17:52:32.687147+0000","last_active":"2024-05-03T17:53:18.968246+0000","last_peered":"2024-05-03T17:53:18.968246+0000","last_clean":"2024-05-03T17:53:18.968246+0000","last_became_active":"2024-05-03T17:52:32.686831+0000","last_became_peered":"2024-05-03T17:52:32.686831+0000","last_unstale":"2024-05-03T17:53:18.968246+0000","last_undegraded":"2024-05-03T17:53:18.968246+0000","last_fullsized":"2024-05-03T17:53:18.968246+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:24:30.092829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.869761+0000","last_change":"2024-05-03T17:52:35.721862+0000","last_active":"2024-05-03T17:53:18.869761+0000","last_peered":"2024-05-03T17:53:18.869761+0000","last_clean":"2024-05-03T17:53:18.869761+0000","last_became_active":"2024-05-03T17:52:35.721393+0000","last_became_peered":"2024-05-03T17:52:35.721393+0000","last_unstale":"2024-05-03T17:53:18.869761+0000","last_undegraded":"2024-05-03T17:53:18.869761+0000","last_fullsized":"2024-05-03T17:53:18.869761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:47:32.014817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885948+0000","last_change":"2024-05-03T17:52:32.687867+0000","last_active":"2024-05-03T17:53:19.885948+0000","last_peered":"2024-05-03T17:53:19.885948+0000","last_clean":"2024-05-03T17:53:19.885948+0000","last_became_active":"2024-05-03T17:52:32.687557+0000","last_became_peered":"2024-05-03T17:52:32.687557+0000","last_unstale":"2024-05-03T17:53:19.885948+0000","last_undegraded":"2024-05-03T17:53:19.885948+0000","last_fullsized":"2024-05-03T17:53:19.885948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:45.863180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968459+0000","last_change":"2024-05-03T17:52:35.731057+0000","last_active":"2024-05-03T17:53:18.968459+0000","last_peered":"2024-05-03T17:53:18.968459+0000","last_clean":"2024-05-03T17:53:18.968459+0000","last_became_active":"2024-05-03T17:52:35.729293+0000","last_became_peered":"2024-05-03T17:52:35.729293+0000","last_unstale":"2024-05-03T17:53:18.968459+0000","last_undegraded":"2024-05-03T17:53:18.968459+0000","last_fullsized":"2024-05-03T17:53:18.968459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:12:53.888078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.872459+0000","last_change":"2024-05-03T17:52:32.689675+0000","last_active":"2024-05-03T17:53:18.872459+0000","last_peered":"2024-05-03T17:53:18.872459+0000","last_clean":"2024-05-03T17:53:18.872459+0000","last_became_active":"2024-05-03T17:52:32.689340+0000","last_became_peered":"2024-05-03T17:52:32.689340+0000","last_unstale":"2024-05-03T17:53:18.872459+0000","last_undegraded":"2024-05-03T17:53:18.872459+0000","last_fullsized":"2024-05-03T17:53:18.872459+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:31.398045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715044+0000","last_change":"2024-05-03T17:52:35.728228+0000","last_active":"2024-05-03T17:52:37.715044+0000","last_peered":"2024-05-03T17:52:37.715044+0000","last_clean":"2024-05-03T17:52:37.715044+0000","last_became_active":"2024-05-03T17:52:35.727233+0000","last_became_peered":"2024-05-03T17:52:35.727233+0000","last_unstale":"2024-05-03T17:52:37.715044+0000","last_undegraded":"2024-05-03T17:52:37.715044+0000","last_fullsized":"2024-05-03T17:52:37.715044+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:57.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886416+0000","last_change":"2024-05-03T17:52:33.249690+0000","last_active":"2024-05-03T17:53:19.886416+0000","last_peered":"2024-05-03T17:53:19.886416+0000","last_clean":"2024-05-03T17:53:19.886416+0000","last_became_active":"2024-05-03T17:52:33.249302+0000","last_became_peered":"2024-05-03T17:52:33.249302+0000","last_unstale":"2024-05-03T17:53:19.886416+0000","last_undegraded":"2024-05-03T17:53:19.886416+0000","last_fullsized":"2024-05-03T17:53:19.886416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:05:48.700670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715448+0000","last_change":"2024-05-03T17:52:35.727171+0000","last_active":"2024-05-03T17:52:37.715448+0000","last_peered":"2024-05-03T17:52:37.715448+0000","last_clean":"2024-05-03T17:52:37.715448+0000","last_became_active":"2024-05-03T17:52:35.726834+0000","last_became_peered":"2024-05-03T17:52:35.726834+0000","last_unstale":"2024-05-03T17:52:37.715448+0000","last_undegraded":"2024-05-03T17:52:37.715448+0000","last_fullsized":"2024-05-03T17:52:37.715448+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:40:09.938183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873627+0000","last_change":"2024-05-03T17:52:32.693694+0000","last_active":"2024-05-03T17:53:18.873627+0000","last_peered":"2024-05-03T17:53:18.873627+0000","last_clean":"2024-05-03T17:53:18.873627+0000","last_became_active":"2024-05-03T17:52:32.693310+0000","last_became_peered":"2024-05-03T17:52:32.693310+0000","last_unstale":"2024-05-03T17:53:18.873627+0000","last_undegraded":"2024-05-03T17:53:18.873627+0000","last_fullsized":"2024-05-03T17:53:18.873627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:59:37.449434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:38.274848+0000","last_change":"2024-05-03T17:52:35.728267+0000","last_active":"2024-05-03T17:52:38.274848+0000","last_peered":"2024-05-03T17:52:38.274848+0000","last_clean":"2024-05-03T17:52:38.274848+0000","last_became_active":"2024-05-03T17:52:35.727902+0000","last_became_peered":"2024-05-03T17:52:35.727902+0000","last_unstale":"2024-05-03T17:52:38.274848+0000","last_undegraded":"2024-05-03T17:52:38.274848+0000","last_fullsized":"2024-05-03T17:52:38.274848+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:09.223850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.729054+0000","last_change":"2024-05-03T17:52:32.687248+0000","last_active":"2024-05-03T17:52:37.729054+0000","last_peered":"2024-05-03T17:52:37.729054+0000","last_clean":"2024-05-03T17:52:37.729054+0000","last_became_active":"2024-05-03T17:52:32.686921+0000","last_became_peered":"2024-05-03T17:52:32.686921+0000","last_unstale":"2024-05-03T17:52:37.729054+0000","last_undegraded":"2024-05-03T17:52:37.729054+0000","last_fullsized":"2024-05-03T17:52:37.729054+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:35:40.849484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715402+0000","last_change":"2024-05-03T17:52:35.729383+0000","last_active":"2024-05-03T17:52:37.715402+0000","last_peered":"2024-05-03T17:52:37.715402+0000","last_clean":"2024-05-03T17:52:37.715402+0000","last_became_active":"2024-05-03T17:52:35.729037+0000","last_became_peered":"2024-05-03T17:52:35.729037+0000","last_unstale":"2024-05-03T17:52:37.715402+0000","last_undegraded":"2024-05-03T17:52:37.715402+0000","last_fullsized":"2024-05-03T17:52:37.715402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:03:03.815770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.869894+0000","last_change":"2024-05-03T17:52:35.731809+0000","last_active":"2024-05-03T17:53:18.869894+0000","last_peered":"2024-05-03T17:53:18.869894+0000","last_clean":"2024-05-03T17:53:18.869894+0000","last_became_active":"2024-05-03T17:52:35.729360+0000","last_became_peered":"2024-05-03T17:52:35.729360+0000","last_unstale":"2024-05-03T17:53:18.869894+0000","last_undegraded":"2024-05-03T17:53:18.869894+0000","last_fullsized":"2024-05-03T17:53:18.869894+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:22.181184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886168+0000","last_change":"2024-05-03T17:52:35.727771+0000","last_active":"2024-05-03T17:53:19.886168+0000","last_peered":"2024-05-03T17:53:19.886168+0000","last_clean":"2024-05-03T17:53:19.886168+0000","last_became_active":"2024-05-03T17:52:35.727414+0000","last_became_peered":"2024-05-03T17:52:35.727414+0000","last_unstale":"2024-05-03T17:53:19.886168+0000","last_undegraded":"2024-05-03T17:53:19.886168+0000","last_fullsized":"2024-05-03T17:53:19.886168+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:09:02.905481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885867+0000","last_change":"2024-05-03T17:52:35.727433+0000","last_active":"2024-05-03T17:53:19.885867+0000","last_peered":"2024-05-03T17:53:19.885867+0000","last_clean":"2024-05-03T17:53:19.885867+0000","last_became_active":"2024-05-03T17:52:35.727123+0000","last_became_peered":"2024-05-03T17:52:35.727123+0000","last_unstale":"2024-05-03T17:53:19.885867+0000","last_undegraded":"2024-05-03T17:53:19.885867+0000","last_fullsized":"2024-05-03T17:53:19.885867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:41:36.871788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722924+0000","last_change":"2024-05-03T17:52:35.721332+0000","last_active":"2024-05-03T17:52:37.722924+0000","last_peered":"2024-05-03T17:52:37.722924+0000","last_clean":"2024-05-03T17:52:37.722924+0000","last_became_active":"2024-05-03T17:52:35.721004+0000","last_became_peered":"2024-05-03T17:52:35.721004+0000","last_unstale":"2024-05-03T17:52:37.722924+0000","last_undegraded":"2024-05-03T17:52:37.722924+0000","last_fullsized":"2024-05-03T17:52:37.722924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:35:03.443163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721268+0000","last_change":"2024-05-03T17:52:35.723649+0000","last_active":"2024-05-03T17:52:37.721268+0000","last_peered":"2024-05-03T17:52:37.721268+0000","last_clean":"2024-05-03T17:52:37.721268+0000","last_became_active":"2024-05-03T17:52:35.723303+0000","last_became_peered":"2024-05-03T17:52:35.723303+0000","last_unstale":"2024-05-03T17:52:37.721268+0000","last_undegraded":"2024-05-03T17:52:37.721268+0000","last_fullsized":"2024-05-03T17:52:37.721268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:02:49.093039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.870949+0000","last_change":"2024-05-03T17:52:35.732505+0000","last_active":"2024-05-03T17:53:18.870949+0000","last_peered":"2024-05-03T17:53:18.870949+0000","last_clean":"2024-05-03T17:53:18.870949+0000","last_became_active":"2024-05-03T17:52:35.732180+0000","last_became_peered":"2024-05-03T17:52:35.732180+0000","last_unstale":"2024-05-03T17:53:18.870949+0000","last_undegraded":"2024-05-03T17:53:18.870949+0000","last_fullsized":"2024-05-03T17:53:18.870949+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.371260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886349+0000","last_change":"2024-05-03T17:52:35.732217+0000","last_active":"2024-05-03T17:53:19.886349+0000","last_peered":"2024-05-03T17:53:19.886349+0000","last_clean":"2024-05-03T17:53:19.886349+0000","last_became_active":"2024-05-03T17:52:35.731868+0000","last_became_peered":"2024-05-03T17:52:35.731868+0000","last_unstale":"2024-05-03T17:53:19.886349+0000","last_undegraded":"2024-05-03T17:53:19.886349+0000","last_fullsized":"2024-05-03T17:53:19.886349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:04:17.521528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968577+0000","last_change":"2024-05-03T17:52:35.723641+0000","last_active":"2024-05-03T17:53:18.968577+0000","last_peered":"2024-05-03T17:53:18.968577+0000","last_clean":"2024-05-03T17:53:18.968577+0000","last_became_active":"2024-05-03T17:52:35.723323+0000","last_became_peered":"2024-05-03T17:52:35.723323+0000","last_unstale":"2024-05-03T17:53:18.968577+0000","last_undegraded":"2024-05-03T17:53:18.968577+0000","last_fullsized":"2024-05-03T17:53:18.968577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:45:34.032407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873223+0000","last_change":"2024-05-03T17:52:35.731008+0000","last_active":"2024-05-03T17:53:18.873223+0000","last_peered":"2024-05-03T17:53:18.873223+0000","last_clean":"2024-05-03T17:53:18.873223+0000","last_became_active":"2024-05-03T17:52:35.730627+0000","last_became_peered":"2024-05-03T17:52:35.730627+0000","last_unstale":"2024-05-03T17:53:18.873223+0000","last_undegraded":"2024-05-03T17:53:18.873223+0000","last_fullsized":"2024-05-03T17:53:18.873223+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:30:32.884518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968191+0000","last_change":"2024-05-03T17:52:35.726938+0000","last_active":"2024-05-03T17:53:18.968191+0000","last_peered":"2024-05-03T17:53:18.968191+0000","last_clean":"2024-05-03T17:53:18.968191+0000","last_became_active":"2024-05-03T17:52:35.726611+0000","last_became_peered":"2024-05-03T17:52:35.726611+0000","last_unstale":"2024-05-03T17:53:18.968191+0000","last_undegraded":"2024-05-03T17:53:18.968191+0000","last_fullsized":"2024-05-03T17:53:18.968191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:11.587035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722196+0000","last_change":"2024-05-03T17:52:35.722562+0000","last_active":"2024-05-03T17:52:37.722196+0000","last_peered":"2024-05-03T17:52:37.722196+0000","last_clean":"2024-05-03T17:52:37.722196+0000","last_became_active":"2024-05-03T17:52:35.722229+0000","last_became_peered":"2024-05-03T17:52:35.722229+0000","last_unstale":"2024-05-03T17:52:37.722196+0000","last_undegraded":"2024-05-03T17:52:37.722196+0000","last_fullsized":"2024-05-03T17:52:37.722196+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:57:06.730533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721608+0000","last_change":"2024-05-03T17:52:35.722707+0000","last_active":"2024-05-03T17:52:37.721608+0000","last_peered":"2024-05-03T17:52:37.721608+0000","last_clean":"2024-05-03T17:52:37.721608+0000","last_became_active":"2024-05-03T17:52:35.721847+0000","last_became_peered":"2024-05-03T17:52:35.721847+0000","last_unstale":"2024-05-03T17:52:37.721608+0000","last_undegraded":"2024-05-03T17:52:37.721608+0000","last_fullsized":"2024-05-03T17:52:37.721608+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:56:21.620074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.870385+0000","last_change":"2024-05-03T17:52:35.731810+0000","last_active":"2024-05-03T17:53:18.870385+0000","last_peered":"2024-05-03T17:53:18.870385+0000","last_clean":"2024-05-03T17:53:18.870385+0000","last_became_active":"2024-05-03T17:52:35.728986+0000","last_became_peered":"2024-05-03T17:52:35.728986+0000","last_unstale":"2024-05-03T17:53:18.870385+0000","last_undegraded":"2024-05-03T17:53:18.870385+0000","last_fullsized":"2024-05-03T17:53:18.870385+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:18.275792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280811,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53993,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":176,"num_read_kb":144,"num_write":157,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":161979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743449,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":839680,"data_stored":409579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906974,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1042525,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070499,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":843776,"data_stored":401288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266728,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":843776,"data_stored":401208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":993529,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":831488,"data_stored":400677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":450071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":851968,"data_stored":401948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":460339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":860160,"data_stored":409515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":847872,"data_stored":410032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":992822,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":49601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":51342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":51570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T17:53:21.819 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T17:53:22.059 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:22.180 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:21 smithi122 ceph-mon[29744]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 5 op/s 2024-05-03T17:53:22.180 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:21 smithi122 ceph-mon[29744]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:22.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:21 smithi149 ceph-mon[33409]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 5 op/s 2024-05-03T17:53:22.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:21 smithi149 ceph-mon[33409]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:22.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:21 smithi192 ceph-mon[33753]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 629 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 5 op/s 2024-05-03T17:53:22.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:21 smithi192 ceph-mon[33753]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:23.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.723+0000 7f5004517700 1 -- 172.21.15.122:0/533525454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ffc102390 msgr2=0x7f4ffc102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:23.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.723+0000 7f5004517700 1 --2- 172.21.15.122:0/533525454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ffc102390 0x7f4ffc102810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009990 tx=0x7f4fec009ca0 comp rx=0 tx=0).stop 2024-05-03T17:53:23.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.723+0000 7f5004517700 1 -- 172.21.15.122:0/533525454 shutdown_connections 2024-05-03T17:53:23.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.723+0000 7f5004517700 1 --2- 172.21.15.122:0/533525454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ffc102d50 0x7f4ffc10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:23.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.723+0000 7f5004517700 1 --2- 172.21.15.122:0/533525454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc108400 0x7f4ffc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:23.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.723+0000 7f5004517700 1 --2- 172.21.15.122:0/533525454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ffc102390 0x7f4ffc102810 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:23.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.723+0000 7f5004517700 1 -- 172.21.15.122:0/533525454 >> 172.21.15.122:0/533525454 conn(0x7f4ffc079310 msgr2=0x7f4ffc079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:23.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.723+0000 7f5004517700 1 -- 172.21.15.122:0/533525454 shutdown_connections 2024-05-03T17:53:23.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.723+0000 7f5004517700 1 -- 172.21.15.122:0/533525454 wait complete. 2024-05-03T17:53:23.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.724+0000 7f5004517700 1 Processor -- start 2024-05-03T17:53:23.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.724+0000 7f5004517700 1 -- start start 2024-05-03T17:53:23.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.724+0000 7f5004517700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ffc102390 0x7f4ffc198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:23.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.724+0000 7f5004517700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ffc102d50 0x7f4ffc198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:23.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.724+0000 7f5004517700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc108400 0x7f4ffc19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:23.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.724+0000 7f5004517700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc112550 con 0x7f4ffc102390 2024-05-03T17:53:23.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.724+0000 7f5004517700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc1123d0 con 0x7f4ffc108400 2024-05-03T17:53:23.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.724+0000 7f5004517700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc1126d0 con 0x7f4ffc102d50 2024-05-03T17:53:23.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.724+0000 7f50022b3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ffc102390 0x7f4ffc198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:23.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f50022b3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ffc102390 0x7f4ffc198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39236/0 (socket says 172.21.15.122:39236) 2024-05-03T17:53:23.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f50022b3700 1 -- 172.21.15.122:0/2223267158 learned_addr learned my addr 172.21.15.122:0/2223267158 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:23.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f5002ab4700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc108400 0x7f4ffc19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:23.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f5001ab2700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ffc102d50 0x7f4ffc198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:23.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f50022b3700 1 -- 172.21.15.122:0/2223267158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ffc102d50 msgr2=0x7f4ffc198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:23.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f50022b3700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ffc102d50 0x7f4ffc198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:23.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f50022b3700 1 -- 172.21.15.122:0/2223267158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc108400 msgr2=0x7f4ffc19d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:23.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f50022b3700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc108400 0x7f4ffc19d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:23.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f50022b3700 1 -- 172.21.15.122:0/2223267158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ffc19dd90 con 0x7f4ffc102390 2024-05-03T17:53:23.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f50022b3700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ffc102390 0x7f4ffc198800 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800e9a0 tx=0x7f4ff800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:23.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f4ff37fe700 1 -- 172.21.15.122:0/2223267158 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800ca10 con 0x7f4ffc102390 2024-05-03T17:53:23.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f5004517700 1 -- 172.21.15.122:0/2223267158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ffc19e080 con 0x7f4ffc102390 2024-05-03T17:53:23.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f4ff37fe700 1 -- 172.21.15.122:0/2223267158 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff800cbb0 con 0x7f4ffc102390 2024-05-03T17:53:23.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f4ff37fe700 1 -- 172.21.15.122:0/2223267158 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8018680 con 0x7f4ffc102390 2024-05-03T17:53:23.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.725+0000 7f5004517700 1 -- 172.21.15.122:0/2223267158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ffc19e590 con 0x7f4ffc102390 2024-05-03T17:53:23.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.727+0000 7f5004517700 1 -- 172.21.15.122:0/2223267158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ffc06a190 con 0x7f4ffc102390 2024-05-03T17:53:23.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.728+0000 7f4ff37fe700 1 -- 172.21.15.122:0/2223267158 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f4ff8010430 con 0x7f4ffc102390 2024-05-03T17:53:23.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.729+0000 7f4ff37fe700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4fe80b2910 0x7f4fe80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:23.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.729+0000 7f5001ab2700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4fe80b2910 0x7f4fe80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:23.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.729+0000 7f4ff37fe700 1 -- 172.21.15.122:0/2223267158 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f4ff80e26b0 con 0x7f4ffc102390 2024-05-03T17:53:23.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.729+0000 7f5001ab2700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4fe80b2910 0x7f4fe80b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00b2e0 tx=0x7f4fec005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:23.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.732+0000 7f4ff37fe700 1 -- 172.21.15.122:0/2223267158 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4ff80aa370 con 0x7f4ffc102390 2024-05-03T17:53:23.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.935+0000 7f5004517700 1 -- 172.21.15.122:0/2223267158 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ffc19e7c0 con 0x7f4fe80b2910 2024-05-03T17:53:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.942+0000 7f4ff37fe700 1 -- 172.21.15.122:0/2223267158 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258874 (secure 0 0 0) 0x7f4fec067350 con 0x7f4fe80b2910 2024-05-03T17:53:23.941 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:53:23.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 -- 172.21.15.122:0/2223267158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4fe80b2910 msgr2=0x7f4fe80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:23.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4fe80b2910 0x7f4fe80b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00b2e0 tx=0x7f4fec005960 comp rx=0 tx=0).stop 2024-05-03T17:53:23.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 -- 172.21.15.122:0/2223267158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ffc102390 msgr2=0x7f4ffc198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:23.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ffc102390 0x7f4ffc198800 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800e9a0 tx=0x7f4ff800ecb0 comp rx=0 tx=0).stop 2024-05-03T17:53:23.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 -- 172.21.15.122:0/2223267158 shutdown_connections 2024-05-03T17:53:23.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4fe80b2910 0x7f4fe80b4dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:23.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc108400 0x7f4ffc19d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:23.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ffc102390 0x7f4ffc198800 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:23.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 --2- 172.21.15.122:0/2223267158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ffc102d50 0x7f4ffc198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:23.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 -- 172.21.15.122:0/2223267158 >> 172.21.15.122:0/2223267158 conn(0x7f4ffc079310 msgr2=0x7f4ffc1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:23.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 -- 172.21.15.122:0/2223267158 shutdown_connections 2024-05-03T17:53:23.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:23.945+0000 7f5004517700 1 -- 172.21.15.122:0/2223267158 wait complete. 2024-05-03T17:53:23.944 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-05-03T17:53:24.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:23 smithi122 ceph-mon[29744]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-03T17:53:24.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:23 smithi149 ceph-mon[33409]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-03T17:53:24.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:23 smithi192 ceph-mon[33753]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 633 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-03T17:53:24.585 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-05-03T17:53:22.548858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648159,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":268,"num_read_kb":222,"num_write":285,"num_write_kb":2255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376136,"kb_used_data":10312,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565688,"statfs":{"total":1151940427776,"available":1151555264512,"internally_reserved":0,"allocated":10559488,"data_stored":6827857,"data_compressed":107130,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3564,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":29,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001995"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.870586+0000","last_change":"2024-05-03T17:52:35.725057+0000","last_active":"2024-05-03T17:53:18.870586+0000","last_peered":"2024-05-03T17:53:18.870586+0000","last_clean":"2024-05-03T17:53:18.870586+0000","last_became_active":"2024-05-03T17:52:35.724632+0000","last_became_peered":"2024-05-03T17:52:35.724632+0000","last_unstale":"2024-05-03T17:53:18.870586+0000","last_undegraded":"2024-05-03T17:53:18.870586+0000","last_fullsized":"2024-05-03T17:53:18.870586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:46.065357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722405+0000","last_change":"2024-05-03T17:52:35.728500+0000","last_active":"2024-05-03T17:52:37.722405+0000","last_peered":"2024-05-03T17:52:37.722405+0000","last_clean":"2024-05-03T17:52:37.722405+0000","last_became_active":"2024-05-03T17:52:35.728210+0000","last_became_peered":"2024-05-03T17:52:35.728210+0000","last_unstale":"2024-05-03T17:52:37.722405+0000","last_undegraded":"2024-05-03T17:52:37.722405+0000","last_fullsized":"2024-05-03T17:52:37.722405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:33:23.410617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722037+0000","last_change":"2024-05-03T17:52:35.726300+0000","last_active":"2024-05-03T17:52:37.722037+0000","last_peered":"2024-05-03T17:52:37.722037+0000","last_clean":"2024-05-03T17:52:37.722037+0000","last_became_active":"2024-05-03T17:52:35.725994+0000","last_became_peered":"2024-05-03T17:52:35.725994+0000","last_unstale":"2024-05-03T17:52:37.722037+0000","last_undegraded":"2024-05-03T17:52:37.722037+0000","last_fullsized":"2024-05-03T17:52:37.722037+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:31:19.723453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.867989+0000","last_change":"2024-05-03T17:52:35.728105+0000","last_active":"2024-05-03T17:53:18.867989+0000","last_peered":"2024-05-03T17:53:18.867989+0000","last_clean":"2024-05-03T17:53:18.867989+0000","last_became_active":"2024-05-03T17:52:35.727762+0000","last_became_peered":"2024-05-03T17:52:35.727762+0000","last_unstale":"2024-05-03T17:53:18.867989+0000","last_undegraded":"2024-05-03T17:53:18.867989+0000","last_fullsized":"2024-05-03T17:53:18.867989+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.640057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.872403+0000","last_change":"2024-05-03T17:52:35.734516+0000","last_active":"2024-05-03T17:53:18.872403+0000","last_peered":"2024-05-03T17:53:18.872403+0000","last_clean":"2024-05-03T17:53:18.872403+0000","last_became_active":"2024-05-03T17:52:35.734125+0000","last_became_peered":"2024-05-03T17:52:35.734125+0000","last_unstale":"2024-05-03T17:53:18.872403+0000","last_undegraded":"2024-05-03T17:53:18.872403+0000","last_fullsized":"2024-05-03T17:53:18.872403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:39:25.103137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871539+0000","last_change":"2024-05-03T17:52:35.723427+0000","last_active":"2024-05-03T17:53:19.871539+0000","last_peered":"2024-05-03T17:53:19.871539+0000","last_clean":"2024-05-03T17:53:19.871539+0000","last_became_active":"2024-05-03T17:52:35.723113+0000","last_became_peered":"2024-05-03T17:52:35.723113+0000","last_unstale":"2024-05-03T17:53:19.871539+0000","last_undegraded":"2024-05-03T17:53:19.871539+0000","last_fullsized":"2024-05-03T17:53:19.871539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:37:14.606347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.714785+0000","last_change":"2024-05-03T17:52:35.730858+0000","last_active":"2024-05-03T17:52:37.714785+0000","last_peered":"2024-05-03T17:52:37.714785+0000","last_clean":"2024-05-03T17:52:37.714785+0000","last_became_active":"2024-05-03T17:52:35.730453+0000","last_became_peered":"2024-05-03T17:52:35.730453+0000","last_unstale":"2024-05-03T17:52:37.714785+0000","last_undegraded":"2024-05-03T17:52:37.714785+0000","last_fullsized":"2024-05-03T17:52:37.714785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:21:17.803027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968183+0000","last_change":"2024-05-03T17:52:35.730547+0000","last_active":"2024-05-03T17:53:18.968183+0000","last_peered":"2024-05-03T17:53:18.968183+0000","last_clean":"2024-05-03T17:53:18.968183+0000","last_became_active":"2024-05-03T17:52:35.729582+0000","last_became_peered":"2024-05-03T17:52:35.729582+0000","last_unstale":"2024-05-03T17:53:18.968183+0000","last_undegraded":"2024-05-03T17:53:18.968183+0000","last_fullsized":"2024-05-03T17:53:18.968183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:24:55.110537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.868010+0000","last_change":"2024-05-03T17:52:35.726305+0000","last_active":"2024-05-03T17:53:19.868010+0000","last_peered":"2024-05-03T17:53:19.868010+0000","last_clean":"2024-05-03T17:53:19.868010+0000","last_became_active":"2024-05-03T17:52:35.725858+0000","last_became_peered":"2024-05-03T17:52:35.725858+0000","last_unstale":"2024-05-03T17:53:19.868010+0000","last_undegraded":"2024-05-03T17:53:19.868010+0000","last_fullsized":"2024-05-03T17:53:19.868010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:59.347307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871846+0000","last_change":"2024-05-03T17:52:35.726865+0000","last_active":"2024-05-03T17:53:19.871846+0000","last_peered":"2024-05-03T17:53:19.871846+0000","last_clean":"2024-05-03T17:53:19.871846+0000","last_became_active":"2024-05-03T17:52:35.726483+0000","last_became_peered":"2024-05-03T17:52:35.726483+0000","last_unstale":"2024-05-03T17:53:19.871846+0000","last_undegraded":"2024-05-03T17:53:19.871846+0000","last_fullsized":"2024-05-03T17:53:19.871846+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:17.851133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715343+0000","last_change":"2024-05-03T17:52:35.730551+0000","last_active":"2024-05-03T17:52:37.715343+0000","last_peered":"2024-05-03T17:52:37.715343+0000","last_clean":"2024-05-03T17:52:37.715343+0000","last_became_active":"2024-05-03T17:52:35.730187+0000","last_became_peered":"2024-05-03T17:52:35.730187+0000","last_unstale":"2024-05-03T17:52:37.715343+0000","last_undegraded":"2024-05-03T17:52:37.715343+0000","last_fullsized":"2024-05-03T17:52:37.715343+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:18:56.211061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.867924+0000","last_change":"2024-05-03T17:52:35.730040+0000","last_active":"2024-05-03T17:53:18.867924+0000","last_peered":"2024-05-03T17:53:18.867924+0000","last_clean":"2024-05-03T17:53:18.867924+0000","last_became_active":"2024-05-03T17:52:35.729549+0000","last_became_peered":"2024-05-03T17:52:35.729549+0000","last_unstale":"2024-05-03T17:53:18.867924+0000","last_undegraded":"2024-05-03T17:53:18.867924+0000","last_fullsized":"2024-05-03T17:53:18.867924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:50:05.052247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721722+0000","last_change":"2024-05-03T17:52:35.726709+0000","last_active":"2024-05-03T17:52:37.721722+0000","last_peered":"2024-05-03T17:52:37.721722+0000","last_clean":"2024-05-03T17:52:37.721722+0000","last_became_active":"2024-05-03T17:52:35.726166+0000","last_became_peered":"2024-05-03T17:52:35.726166+0000","last_unstale":"2024-05-03T17:52:37.721722+0000","last_undegraded":"2024-05-03T17:52:37.721722+0000","last_fullsized":"2024-05-03T17:52:37.721722+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.164622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722305+0000","last_change":"2024-05-03T17:52:35.727583+0000","last_active":"2024-05-03T17:52:37.722305+0000","last_peered":"2024-05-03T17:52:37.722305+0000","last_clean":"2024-05-03T17:52:37.722305+0000","last_became_active":"2024-05-03T17:52:35.727131+0000","last_became_peered":"2024-05-03T17:52:35.727131+0000","last_unstale":"2024-05-03T17:52:37.722305+0000","last_undegraded":"2024-05-03T17:52:37.722305+0000","last_fullsized":"2024-05-03T17:52:37.722305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:28:21.255954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721652+0000","last_change":"2024-05-03T17:52:35.726253+0000","last_active":"2024-05-03T17:52:37.721652+0000","last_peered":"2024-05-03T17:52:37.721652+0000","last_clean":"2024-05-03T17:52:37.721652+0000","last_became_active":"2024-05-03T17:52:35.725491+0000","last_became_peered":"2024-05-03T17:52:35.725491+0000","last_unstale":"2024-05-03T17:52:37.721652+0000","last_undegraded":"2024-05-03T17:52:37.721652+0000","last_fullsized":"2024-05-03T17:52:37.721652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:52:28.424299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867360+0000","last_change":"2024-05-03T17:52:35.717354+0000","last_active":"2024-05-03T17:53:19.867360+0000","last_peered":"2024-05-03T17:53:19.867360+0000","last_clean":"2024-05-03T17:53:19.867360+0000","last_became_active":"2024-05-03T17:52:35.717025+0000","last_became_peered":"2024-05-03T17:52:35.717025+0000","last_unstale":"2024-05-03T17:53:19.867360+0000","last_undegraded":"2024-05-03T17:53:19.867360+0000","last_fullsized":"2024-05-03T17:53:19.867360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:11:07.778484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.870767+0000","last_change":"2024-05-03T17:52:35.725626+0000","last_active":"2024-05-03T17:53:18.870767+0000","last_peered":"2024-05-03T17:53:18.870767+0000","last_clean":"2024-05-03T17:53:18.870767+0000","last_became_active":"2024-05-03T17:52:35.725239+0000","last_became_peered":"2024-05-03T17:52:35.725239+0000","last_unstale":"2024-05-03T17:53:18.870767+0000","last_undegraded":"2024-05-03T17:53:18.870767+0000","last_fullsized":"2024-05-03T17:53:18.870767+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:44:18.359484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869622+0000","last_change":"2024-05-03T17:52:35.724250+0000","last_active":"2024-05-03T17:53:19.869622+0000","last_peered":"2024-05-03T17:53:19.869622+0000","last_clean":"2024-05-03T17:53:19.869622+0000","last_became_active":"2024-05-03T17:52:35.723998+0000","last_became_peered":"2024-05-03T17:52:35.723998+0000","last_unstale":"2024-05-03T17:53:19.869622+0000","last_undegraded":"2024-05-03T17:53:19.869622+0000","last_fullsized":"2024-05-03T17:53:19.869622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:03:32.939559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886262+0000","last_change":"2024-05-03T17:52:35.726237+0000","last_active":"2024-05-03T17:53:19.886262+0000","last_peered":"2024-05-03T17:53:19.886262+0000","last_clean":"2024-05-03T17:53:19.886262+0000","last_became_active":"2024-05-03T17:52:35.725847+0000","last_became_peered":"2024-05-03T17:52:35.725847+0000","last_unstale":"2024-05-03T17:53:19.886262+0000","last_undegraded":"2024-05-03T17:53:19.886262+0000","last_fullsized":"2024-05-03T17:53:19.886262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:06.472532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869723+0000","last_change":"2024-05-03T17:52:32.687198+0000","last_active":"2024-05-03T17:53:19.869723+0000","last_peered":"2024-05-03T17:53:19.869723+0000","last_clean":"2024-05-03T17:53:19.869723+0000","last_became_active":"2024-05-03T17:52:32.686848+0000","last_became_peered":"2024-05-03T17:52:32.686848+0000","last_unstale":"2024-05-03T17:53:19.869723+0000","last_undegraded":"2024-05-03T17:53:19.869723+0000","last_fullsized":"2024-05-03T17:53:19.869723+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:08:24.244371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968473+0000","last_change":"2024-05-03T17:52:35.730595+0000","last_active":"2024-05-03T17:53:18.968473+0000","last_peered":"2024-05-03T17:53:18.968473+0000","last_clean":"2024-05-03T17:53:18.968473+0000","last_became_active":"2024-05-03T17:52:35.729999+0000","last_became_peered":"2024-05-03T17:52:35.729999+0000","last_unstale":"2024-05-03T17:53:18.968473+0000","last_undegraded":"2024-05-03T17:53:18.968473+0000","last_fullsized":"2024-05-03T17:53:18.968473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:14.485756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968634+0000","last_change":"2024-05-03T17:52:32.935239+0000","last_active":"2024-05-03T17:53:18.968634+0000","last_peered":"2024-05-03T17:53:18.968634+0000","last_clean":"2024-05-03T17:53:18.968634+0000","last_became_active":"2024-05-03T17:52:32.934915+0000","last_became_peered":"2024-05-03T17:52:32.934915+0000","last_unstale":"2024-05-03T17:53:18.968634+0000","last_undegraded":"2024-05-03T17:53:18.968634+0000","last_fullsized":"2024-05-03T17:53:18.968634+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:20.785071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.723294+0000","last_change":"2024-05-03T17:52:35.724940+0000","last_active":"2024-05-03T17:52:37.723294+0000","last_peered":"2024-05-03T17:52:37.723294+0000","last_clean":"2024-05-03T17:52:37.723294+0000","last_became_active":"2024-05-03T17:52:35.724603+0000","last_became_peered":"2024-05-03T17:52:35.724603+0000","last_unstale":"2024-05-03T17:52:37.723294+0000","last_undegraded":"2024-05-03T17:52:37.723294+0000","last_fullsized":"2024-05-03T17:52:37.723294+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:43.640499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873517+0000","last_change":"2024-05-03T17:52:32.688202+0000","last_active":"2024-05-03T17:53:18.873517+0000","last_peered":"2024-05-03T17:53:18.873517+0000","last_clean":"2024-05-03T17:53:18.873517+0000","last_became_active":"2024-05-03T17:52:32.687893+0000","last_became_peered":"2024-05-03T17:52:32.687893+0000","last_unstale":"2024-05-03T17:53:18.873517+0000","last_undegraded":"2024-05-03T17:53:18.873517+0000","last_fullsized":"2024-05-03T17:53:18.873517+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:43.342832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867555+0000","last_change":"2024-05-03T17:52:35.720980+0000","last_active":"2024-05-03T17:53:19.867555+0000","last_peered":"2024-05-03T17:53:19.867555+0000","last_clean":"2024-05-03T17:53:19.867555+0000","last_became_active":"2024-05-03T17:52:35.720582+0000","last_became_peered":"2024-05-03T17:52:35.720582+0000","last_unstale":"2024-05-03T17:53:19.867555+0000","last_undegraded":"2024-05-03T17:53:19.867555+0000","last_fullsized":"2024-05-03T17:53:19.867555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:01:22.938186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":2,"reported_epoch":78,"state":"peering","last_fresh":"2024-05-03T17:52:37.715245+0000","last_change":"2024-05-03T17:53:19.412769+0000","last_active":"2024-05-03T17:53:19.412769+0000","last_peered":"2024-05-03T17:52:37.715245+0000","last_clean":"2024-05-03T17:52:37.715245+0000","last_became_active":"2024-05-03T17:52:32.688073+0000","last_became_peered":"2024-05-03T17:52:32.688073+0000","last_unstale":"2024-05-03T17:52:37.715245+0000","last_undegraded":"2024-05-03T17:52:37.715245+0000","last_fullsized":"2024-05-03T17:52:37.715245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.079283+0000","last_change":"2024-05-03T17:53:20.078906+0000","last_active":"2024-05-03T17:53:20.079283+0000","last_peered":"2024-05-03T17:53:20.079283+0000","last_clean":"2024-05-03T17:53:20.079283+0000","last_became_active":"2024-05-03T17:53:20.078538+0000","last_became_peered":"2024-05-03T17:53:20.078538+0000","last_unstale":"2024-05-03T17:53:20.079283+0000","last_undegraded":"2024-05-03T17:53:20.079283+0000","last_fullsized":"2024-05-03T17:53:20.079283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:59:46.945417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873688+0000","last_change":"2024-05-03T17:52:32.692735+0000","last_active":"2024-05-03T17:53:18.873688+0000","last_peered":"2024-05-03T17:53:18.873688+0000","last_clean":"2024-05-03T17:53:18.873688+0000","last_became_active":"2024-05-03T17:52:32.692394+0000","last_became_peered":"2024-05-03T17:52:32.692394+0000","last_unstale":"2024-05-03T17:53:18.873688+0000","last_undegraded":"2024-05-03T17:53:18.873688+0000","last_fullsized":"2024-05-03T17:53:18.873688+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:40:11.434751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.872637+0000","last_change":"2024-05-03T17:52:35.727167+0000","last_active":"2024-05-03T17:53:18.872637+0000","last_peered":"2024-05-03T17:53:18.872637+0000","last_clean":"2024-05-03T17:53:18.872637+0000","last_became_active":"2024-05-03T17:52:35.726792+0000","last_became_peered":"2024-05-03T17:52:35.726792+0000","last_unstale":"2024-05-03T17:53:18.872637+0000","last_undegraded":"2024-05-03T17:53:18.872637+0000","last_fullsized":"2024-05-03T17:53:18.872637+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:54:01.823025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871452+0000","last_change":"2024-05-03T17:52:32.687573+0000","last_active":"2024-05-03T17:53:19.871452+0000","last_peered":"2024-05-03T17:53:19.871452+0000","last_clean":"2024-05-03T17:53:19.871452+0000","last_became_active":"2024-05-03T17:52:32.686158+0000","last_became_peered":"2024-05-03T17:52:32.686158+0000","last_unstale":"2024-05-03T17:53:19.871452+0000","last_undegraded":"2024-05-03T17:53:19.871452+0000","last_fullsized":"2024-05-03T17:53:19.871452+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:19.891479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873865+0000","last_change":"2024-05-03T17:52:35.728049+0000","last_active":"2024-05-03T17:53:18.873865+0000","last_peered":"2024-05-03T17:53:18.873865+0000","last_clean":"2024-05-03T17:53:18.873865+0000","last_became_active":"2024-05-03T17:52:35.727713+0000","last_became_peered":"2024-05-03T17:52:35.727713+0000","last_unstale":"2024-05-03T17:53:18.873865+0000","last_undegraded":"2024-05-03T17:53:18.873865+0000","last_fullsized":"2024-05-03T17:53:18.873865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:28:01.798457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:50.788759+0000","last_change":"2024-05-03T17:52:32.685679+0000","last_active":"2024-05-03T17:52:50.788759+0000","last_peered":"2024-05-03T17:52:50.788759+0000","last_clean":"2024-05-03T17:52:50.788759+0000","last_became_active":"2024-05-03T17:52:32.685306+0000","last_became_peered":"2024-05-03T17:52:32.685306+0000","last_unstale":"2024-05-03T17:52:50.788759+0000","last_undegraded":"2024-05-03T17:52:50.788759+0000","last_fullsized":"2024-05-03T17:52:50.788759+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:48:22.218310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968144+0000","last_change":"2024-05-03T17:52:35.717111+0000","last_active":"2024-05-03T17:53:18.968144+0000","last_peered":"2024-05-03T17:53:18.968144+0000","last_clean":"2024-05-03T17:53:18.968144+0000","last_became_active":"2024-05-03T17:52:35.716782+0000","last_became_peered":"2024-05-03T17:52:35.716782+0000","last_unstale":"2024-05-03T17:53:18.968144+0000","last_undegraded":"2024-05-03T17:53:18.968144+0000","last_fullsized":"2024-05-03T17:53:18.968144+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:31:57.766927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873103+0000","last_change":"2024-05-03T17:52:32.690846+0000","last_active":"2024-05-03T17:53:18.873103+0000","last_peered":"2024-05-03T17:53:18.873103+0000","last_clean":"2024-05-03T17:53:18.873103+0000","last_became_active":"2024-05-03T17:52:32.690506+0000","last_became_peered":"2024-05-03T17:52:32.690506+0000","last_unstale":"2024-05-03T17:53:18.873103+0000","last_undegraded":"2024-05-03T17:53:18.873103+0000","last_fullsized":"2024-05-03T17:53:18.873103+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:19.810205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869511+0000","last_change":"2024-05-03T17:52:35.718229+0000","last_active":"2024-05-03T17:53:19.869511+0000","last_peered":"2024-05-03T17:53:19.869511+0000","last_clean":"2024-05-03T17:53:19.869511+0000","last_became_active":"2024-05-03T17:52:35.717907+0000","last_became_peered":"2024-05-03T17:52:35.717907+0000","last_unstale":"2024-05-03T17:53:19.869511+0000","last_undegraded":"2024-05-03T17:53:19.869511+0000","last_fullsized":"2024-05-03T17:53:19.869511+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:45.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":58,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.805562+0000","last_change":"2024-05-03T17:52:32.691174+0000","last_active":"2024-05-03T17:53:20.805562+0000","last_peered":"2024-05-03T17:53:20.805562+0000","last_clean":"2024-05-03T17:53:20.805562+0000","last_became_active":"2024-05-03T17:52:32.689907+0000","last_became_peered":"2024-05-03T17:52:32.689907+0000","last_unstale":"2024-05-03T17:53:20.805562+0000","last_undegraded":"2024-05-03T17:53:20.805562+0000","last_fullsized":"2024-05-03T17:53:20.805562+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:46.630436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867946+0000","last_change":"2024-05-03T17:52:35.720290+0000","last_active":"2024-05-03T17:53:19.867946+0000","last_peered":"2024-05-03T17:53:19.867946+0000","last_clean":"2024-05-03T17:53:19.867946+0000","last_became_active":"2024-05-03T17:52:35.719962+0000","last_became_peered":"2024-05-03T17:52:35.719962+0000","last_unstale":"2024-05-03T17:53:19.867946+0000","last_undegraded":"2024-05-03T17:53:19.867946+0000","last_fullsized":"2024-05-03T17:53:19.867946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:31:41.301592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:49.781041+0000","last_change":"2024-05-03T17:52:32.689937+0000","last_active":"2024-05-03T17:52:49.781041+0000","last_peered":"2024-05-03T17:52:49.781041+0000","last_clean":"2024-05-03T17:52:49.781041+0000","last_became_active":"2024-05-03T17:52:32.689624+0000","last_became_peered":"2024-05-03T17:52:32.689624+0000","last_unstale":"2024-05-03T17:52:49.781041+0000","last_undegraded":"2024-05-03T17:52:49.781041+0000","last_fullsized":"2024-05-03T17:52:49.781041+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:24.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.723130+0000","last_change":"2024-05-03T17:52:35.726824+0000","last_active":"2024-05-03T17:52:37.723130+0000","last_peered":"2024-05-03T17:52:37.723130+0000","last_clean":"2024-05-03T17:52:37.723130+0000","last_became_active":"2024-05-03T17:52:35.726494+0000","last_became_peered":"2024-05-03T17:52:35.726494+0000","last_unstale":"2024-05-03T17:52:37.723130+0000","last_undegraded":"2024-05-03T17:52:37.723130+0000","last_fullsized":"2024-05-03T17:52:37.723130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:41:21.284852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873930+0000","last_change":"2024-05-03T17:52:32.935471+0000","last_active":"2024-05-03T17:53:18.873930+0000","last_peered":"2024-05-03T17:53:18.873930+0000","last_clean":"2024-05-03T17:53:18.873930+0000","last_became_active":"2024-05-03T17:52:32.935012+0000","last_became_peered":"2024-05-03T17:52:32.935012+0000","last_unstale":"2024-05-03T17:53:18.873930+0000","last_undegraded":"2024-05-03T17:53:18.873930+0000","last_fullsized":"2024-05-03T17:53:18.873930+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:35:38.654577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867634+0000","last_change":"2024-05-03T17:52:35.727124+0000","last_active":"2024-05-03T17:53:19.867634+0000","last_peered":"2024-05-03T17:53:19.867634+0000","last_clean":"2024-05-03T17:53:19.867634+0000","last_became_active":"2024-05-03T17:52:35.726804+0000","last_became_peered":"2024-05-03T17:52:35.726804+0000","last_unstale":"2024-05-03T17:53:19.867634+0000","last_undegraded":"2024-05-03T17:53:19.867634+0000","last_fullsized":"2024-05-03T17:53:19.867634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:32.000803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'28","reported_seq":120,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:21.807474+0000","last_change":"2024-05-03T17:52:32.685014+0000","last_active":"2024-05-03T17:53:21.807474+0000","last_peered":"2024-05-03T17:53:21.807474+0000","last_clean":"2024-05-03T17:53:21.807474+0000","last_became_active":"2024-05-03T17:52:32.684660+0000","last_became_peered":"2024-05-03T17:52:32.684660+0000","last_unstale":"2024-05-03T17:53:21.807474+0000","last_undegraded":"2024-05-03T17:53:21.807474+0000","last_fullsized":"2024-05-03T17:53:21.807474+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:25:56.890283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53165,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":84,"num_write":29,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873225+0000","last_change":"2024-05-03T17:52:35.733675+0000","last_active":"2024-05-03T17:53:18.873225+0000","last_peered":"2024-05-03T17:53:18.873225+0000","last_clean":"2024-05-03T17:53:18.873225+0000","last_became_active":"2024-05-03T17:52:35.733234+0000","last_became_peered":"2024-05-03T17:52:35.733234+0000","last_unstale":"2024-05-03T17:53:18.873225+0000","last_undegraded":"2024-05-03T17:53:18.873225+0000","last_fullsized":"2024-05-03T17:53:18.873225+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:57.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886182+0000","last_change":"2024-05-03T17:52:32.690462+0000","last_active":"2024-05-03T17:53:19.886182+0000","last_peered":"2024-05-03T17:53:19.886182+0000","last_clean":"2024-05-03T17:53:19.886182+0000","last_became_active":"2024-05-03T17:52:32.690169+0000","last_became_peered":"2024-05-03T17:52:32.690169+0000","last_unstale":"2024-05-03T17:53:19.886182+0000","last_undegraded":"2024-05-03T17:53:19.886182+0000","last_fullsized":"2024-05-03T17:53:19.886182+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:59.324612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721144+0000","last_change":"2024-05-03T17:52:35.720730+0000","last_active":"2024-05-03T17:52:37.721144+0000","last_peered":"2024-05-03T17:52:37.721144+0000","last_clean":"2024-05-03T17:52:37.721144+0000","last_became_active":"2024-05-03T17:52:35.720392+0000","last_became_peered":"2024-05-03T17:52:35.720392+0000","last_unstale":"2024-05-03T17:52:37.721144+0000","last_undegraded":"2024-05-03T17:52:37.721144+0000","last_fullsized":"2024-05-03T17:52:37.721144+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:58.528148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873704+0000","last_change":"2024-05-03T17:52:32.691439+0000","last_active":"2024-05-03T17:53:18.873704+0000","last_peered":"2024-05-03T17:53:18.873704+0000","last_clean":"2024-05-03T17:53:18.873704+0000","last_became_active":"2024-05-03T17:52:32.691080+0000","last_became_peered":"2024-05-03T17:52:32.691080+0000","last_unstale":"2024-05-03T17:53:18.873704+0000","last_undegraded":"2024-05-03T17:53:18.873704+0000","last_fullsized":"2024-05-03T17:53:18.873704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:40:43.120194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715331+0000","last_change":"2024-05-03T17:52:35.729860+0000","last_active":"2024-05-03T17:52:37.715331+0000","last_peered":"2024-05-03T17:52:37.715331+0000","last_clean":"2024-05-03T17:52:37.715331+0000","last_became_active":"2024-05-03T17:52:35.729508+0000","last_became_peered":"2024-05-03T17:52:35.729508+0000","last_unstale":"2024-05-03T17:52:37.715331+0000","last_undegraded":"2024-05-03T17:52:37.715331+0000","last_fullsized":"2024-05-03T17:52:37.715331+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:39:20.859215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.868360+0000","last_change":"2024-05-03T17:52:32.689346+0000","last_active":"2024-05-03T17:53:18.868360+0000","last_peered":"2024-05-03T17:53:18.868360+0000","last_clean":"2024-05-03T17:53:18.868360+0000","last_became_active":"2024-05-03T17:52:32.689009+0000","last_became_peered":"2024-05-03T17:52:32.689009+0000","last_unstale":"2024-05-03T17:53:18.868360+0000","last_undegraded":"2024-05-03T17:53:18.868360+0000","last_fullsized":"2024-05-03T17:53:18.868360+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:55:36.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.714758+0000","last_change":"2024-05-03T17:52:35.729226+0000","last_active":"2024-05-03T17:52:37.714758+0000","last_peered":"2024-05-03T17:52:37.714758+0000","last_clean":"2024-05-03T17:52:37.714758+0000","last_became_active":"2024-05-03T17:52:35.728846+0000","last_became_peered":"2024-05-03T17:52:35.728846+0000","last_unstale":"2024-05-03T17:52:37.714758+0000","last_undegraded":"2024-05-03T17:52:37.714758+0000","last_fullsized":"2024-05-03T17:52:37.714758+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:39:16.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":39,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.872871+0000","last_change":"2024-05-03T17:52:32.936108+0000","last_active":"2024-05-03T17:53:18.872871+0000","last_peered":"2024-05-03T17:53:18.872871+0000","last_clean":"2024-05-03T17:53:18.872871+0000","last_became_active":"2024-05-03T17:52:32.935782+0000","last_became_peered":"2024-05-03T17:52:32.935782+0000","last_unstale":"2024-05-03T17:53:18.872871+0000","last_undegraded":"2024-05-03T17:53:18.872871+0000","last_fullsized":"2024-05-03T17:53:18.872871+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:47.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.874458+0000","last_change":"2024-05-03T17:52:35.727163+0000","last_active":"2024-05-03T17:53:18.874458+0000","last_peered":"2024-05-03T17:53:18.874458+0000","last_clean":"2024-05-03T17:53:18.874458+0000","last_became_active":"2024-05-03T17:52:35.726824+0000","last_became_peered":"2024-05-03T17:52:35.726824+0000","last_unstale":"2024-05-03T17:53:18.874458+0000","last_undegraded":"2024-05-03T17:53:18.874458+0000","last_fullsized":"2024-05-03T17:53:18.874458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:24:50.803860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873246+0000","last_change":"2024-05-03T17:52:32.687329+0000","last_active":"2024-05-03T17:53:18.873246+0000","last_peered":"2024-05-03T17:53:18.873246+0000","last_clean":"2024-05-03T17:53:18.873246+0000","last_became_active":"2024-05-03T17:52:32.687025+0000","last_became_peered":"2024-05-03T17:52:32.687025+0000","last_unstale":"2024-05-03T17:53:18.873246+0000","last_undegraded":"2024-05-03T17:53:18.873246+0000","last_fullsized":"2024-05-03T17:53:18.873246+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:42:15.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.870328+0000","last_change":"2024-05-03T17:52:35.728797+0000","last_active":"2024-05-03T17:53:18.870328+0000","last_peered":"2024-05-03T17:53:18.870328+0000","last_clean":"2024-05-03T17:53:18.870328+0000","last_became_active":"2024-05-03T17:52:35.728429+0000","last_became_peered":"2024-05-03T17:52:35.728429+0000","last_unstale":"2024-05-03T17:53:18.870328+0000","last_undegraded":"2024-05-03T17:53:18.870328+0000","last_fullsized":"2024-05-03T17:53:18.870328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:19.584593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872508+0000","last_change":"2024-05-03T17:53:19.872508+0000","last_active":"2024-05-03T17:53:19.872508+0000","last_peered":"2024-05-03T17:53:19.872508+0000","last_clean":"2024-05-03T17:53:19.872508+0000","last_became_active":"2024-05-03T17:53:19.872175+0000","last_became_peered":"2024-05-03T17:53:19.872175+0000","last_unstale":"2024-05-03T17:53:19.872508+0000","last_undegraded":"2024-05-03T17:53:19.872508+0000","last_fullsized":"2024-05-03T17:53:19.872508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:24:08.246884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871588+0000","last_change":"2024-05-03T17:52:35.719115+0000","last_active":"2024-05-03T17:53:19.871588+0000","last_peered":"2024-05-03T17:53:19.871588+0000","last_clean":"2024-05-03T17:53:19.871588+0000","last_became_active":"2024-05-03T17:52:35.718054+0000","last_became_peered":"2024-05-03T17:52:35.718054+0000","last_unstale":"2024-05-03T17:53:19.871588+0000","last_undegraded":"2024-05-03T17:53:19.871588+0000","last_fullsized":"2024-05-03T17:53:19.871588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.903766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715066+0000","last_change":"2024-05-03T17:52:32.688993+0000","last_active":"2024-05-03T17:52:37.715066+0000","last_peered":"2024-05-03T17:52:37.715066+0000","last_clean":"2024-05-03T17:52:37.715066+0000","last_became_active":"2024-05-03T17:52:32.688706+0000","last_became_peered":"2024-05-03T17:52:32.688706+0000","last_unstale":"2024-05-03T17:52:37.715066+0000","last_undegraded":"2024-05-03T17:52:37.715066+0000","last_fullsized":"2024-05-03T17:52:37.715066+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:40:14.023338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872957+0000","last_change":"2024-05-03T17:53:19.872957+0000","last_active":"2024-05-03T17:53:19.872957+0000","last_peered":"2024-05-03T17:53:19.872957+0000","last_clean":"2024-05-03T17:53:19.872957+0000","last_became_active":"2024-05-03T17:53:19.872690+0000","last_became_peered":"2024-05-03T17:53:19.872690+0000","last_unstale":"2024-05-03T17:53:19.872957+0000","last_undegraded":"2024-05-03T17:53:19.872957+0000","last_fullsized":"2024-05-03T17:53:19.872957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:53:43.801983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873912+0000","last_change":"2024-05-03T17:52:35.728958+0000","last_active":"2024-05-03T17:53:18.873912+0000","last_peered":"2024-05-03T17:53:18.873912+0000","last_clean":"2024-05-03T17:53:18.873912+0000","last_became_active":"2024-05-03T17:52:35.728619+0000","last_became_peered":"2024-05-03T17:52:35.728619+0000","last_unstale":"2024-05-03T17:53:18.873912+0000","last_undegraded":"2024-05-03T17:53:18.873912+0000","last_fullsized":"2024-05-03T17:53:18.873912+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:29:29.922324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:53:16.800156+0000","last_change":"2024-05-03T17:52:32.935367+0000","last_active":"2024-05-03T17:53:16.800156+0000","last_peered":"2024-05-03T17:53:16.800156+0000","last_clean":"2024-05-03T17:53:16.800156+0000","last_became_active":"2024-05-03T17:52:32.934938+0000","last_became_peered":"2024-05-03T17:52:32.934938+0000","last_unstale":"2024-05-03T17:53:16.800156+0000","last_undegraded":"2024-05-03T17:53:16.800156+0000","last_fullsized":"2024-05-03T17:53:16.800156+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:07:21.642883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968345+0000","last_change":"2024-05-03T17:52:35.731035+0000","last_active":"2024-05-03T17:53:18.968345+0000","last_peered":"2024-05-03T17:53:18.968345+0000","last_clean":"2024-05-03T17:53:18.968345+0000","last_became_active":"2024-05-03T17:52:35.730034+0000","last_became_peered":"2024-05-03T17:52:35.730034+0000","last_unstale":"2024-05-03T17:53:18.968345+0000","last_undegraded":"2024-05-03T17:53:18.968345+0000","last_fullsized":"2024-05-03T17:53:18.968345+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:25.545290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.867900+0000","last_change":"2024-05-03T17:52:32.690043+0000","last_active":"2024-05-03T17:53:18.867900+0000","last_peered":"2024-05-03T17:53:18.867900+0000","last_clean":"2024-05-03T17:53:18.867900+0000","last_became_active":"2024-05-03T17:52:32.689562+0000","last_became_peered":"2024-05-03T17:52:32.689562+0000","last_unstale":"2024-05-03T17:53:18.867900+0000","last_undegraded":"2024-05-03T17:53:18.867900+0000","last_fullsized":"2024-05-03T17:53:18.867900+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:59.014968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":93,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.714703+0000","last_change":"2024-05-03T17:51:02.340854+0000","last_active":"2024-05-03T17:52:37.714703+0000","last_peered":"2024-05-03T17:52:37.714703+0000","last_clean":"2024-05-03T17:52:37.714703+0000","last_became_active":"2024-05-03T17:51:02.336454+0000","last_became_peered":"2024-05-03T17:51:02.336454+0000","last_unstale":"2024-05-03T17:52:37.714703+0000","last_undegraded":"2024-05-03T17:52:37.714703+0000","last_fullsized":"2024-05-03T17:52:37.714703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_clean_scrub_stamp":"2024-05-03T17:47:41.306771+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:18:08.512098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.720611+0000","last_change":"2024-05-03T17:52:35.727303+0000","last_active":"2024-05-03T17:52:37.720611+0000","last_peered":"2024-05-03T17:52:37.720611+0000","last_clean":"2024-05-03T17:52:37.720611+0000","last_became_active":"2024-05-03T17:52:35.726951+0000","last_became_peered":"2024-05-03T17:52:35.726951+0000","last_unstale":"2024-05-03T17:52:37.720611+0000","last_undegraded":"2024-05-03T17:52:37.720611+0000","last_fullsized":"2024-05-03T17:52:37.720611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:20:31.650582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:50.789304+0000","last_change":"2024-05-03T17:52:32.690093+0000","last_active":"2024-05-03T17:52:50.789304+0000","last_peered":"2024-05-03T17:52:50.789304+0000","last_clean":"2024-05-03T17:52:50.789304+0000","last_became_active":"2024-05-03T17:52:32.689777+0000","last_became_peered":"2024-05-03T17:52:32.689777+0000","last_unstale":"2024-05-03T17:52:50.789304+0000","last_undegraded":"2024-05-03T17:52:50.789304+0000","last_fullsized":"2024-05-03T17:52:50.789304+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:09.693011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873331+0000","last_change":"2024-05-03T17:52:32.935589+0000","last_active":"2024-05-03T17:53:18.873331+0000","last_peered":"2024-05-03T17:53:18.873331+0000","last_clean":"2024-05-03T17:53:18.873331+0000","last_became_active":"2024-05-03T17:52:32.935221+0000","last_became_peered":"2024-05-03T17:52:32.935221+0000","last_unstale":"2024-05-03T17:53:18.873331+0000","last_undegraded":"2024-05-03T17:53:18.873331+0000","last_fullsized":"2024-05-03T17:53:18.873331+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:26:08.001761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721113+0000","last_change":"2024-05-03T17:52:35.724293+0000","last_active":"2024-05-03T17:52:37.721113+0000","last_peered":"2024-05-03T17:52:37.721113+0000","last_clean":"2024-05-03T17:52:37.721113+0000","last_became_active":"2024-05-03T17:52:35.723978+0000","last_became_peered":"2024-05-03T17:52:35.723978+0000","last_unstale":"2024-05-03T17:52:37.721113+0000","last_undegraded":"2024-05-03T17:52:37.721113+0000","last_fullsized":"2024-05-03T17:52:37.721113+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:23.357872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869626+0000","last_change":"2024-05-03T17:52:32.687303+0000","last_active":"2024-05-03T17:53:19.869626+0000","last_peered":"2024-05-03T17:53:19.869626+0000","last_clean":"2024-05-03T17:53:19.869626+0000","last_became_active":"2024-05-03T17:52:32.686827+0000","last_became_peered":"2024-05-03T17:52:32.686827+0000","last_unstale":"2024-05-03T17:53:19.869626+0000","last_undegraded":"2024-05-03T17:53:19.869626+0000","last_fullsized":"2024-05-03T17:53:19.869626+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:11:53.565215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968621+0000","last_change":"2024-05-03T17:52:35.720898+0000","last_active":"2024-05-03T17:53:18.968621+0000","last_peered":"2024-05-03T17:53:18.968621+0000","last_clean":"2024-05-03T17:53:18.968621+0000","last_became_active":"2024-05-03T17:52:35.720572+0000","last_became_peered":"2024-05-03T17:52:35.720572+0000","last_unstale":"2024-05-03T17:53:18.968621+0000","last_undegraded":"2024-05-03T17:53:18.968621+0000","last_fullsized":"2024-05-03T17:53:18.968621+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:01.697167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.868053+0000","last_change":"2024-05-03T17:52:33.249970+0000","last_active":"2024-05-03T17:53:18.868053+0000","last_peered":"2024-05-03T17:53:18.868053+0000","last_clean":"2024-05-03T17:53:18.868053+0000","last_became_active":"2024-05-03T17:52:33.249596+0000","last_became_peered":"2024-05-03T17:52:33.249596+0000","last_unstale":"2024-05-03T17:53:18.868053+0000","last_undegraded":"2024-05-03T17:53:18.868053+0000","last_fullsized":"2024-05-03T17:53:18.868053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:46:53.058586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722553+0000","last_change":"2024-05-03T17:52:35.722129+0000","last_active":"2024-05-03T17:52:37.722553+0000","last_peered":"2024-05-03T17:52:37.722553+0000","last_clean":"2024-05-03T17:52:37.722553+0000","last_became_active":"2024-05-03T17:52:35.721815+0000","last_became_peered":"2024-05-03T17:52:35.721815+0000","last_unstale":"2024-05-03T17:52:37.722553+0000","last_undegraded":"2024-05-03T17:52:37.722553+0000","last_fullsized":"2024-05-03T17:52:37.722553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:33.781350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867376+0000","last_change":"2024-05-03T17:52:32.936157+0000","last_active":"2024-05-03T17:53:19.867376+0000","last_peered":"2024-05-03T17:53:19.867376+0000","last_clean":"2024-05-03T17:53:19.867376+0000","last_became_active":"2024-05-03T17:52:32.935842+0000","last_became_peered":"2024-05-03T17:52:32.935842+0000","last_unstale":"2024-05-03T17:53:19.867376+0000","last_undegraded":"2024-05-03T17:53:19.867376+0000","last_fullsized":"2024-05-03T17:53:19.867376+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:18.006451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.872881+0000","last_change":"2024-05-03T17:52:35.735993+0000","last_active":"2024-05-03T17:53:18.872881+0000","last_peered":"2024-05-03T17:53:18.872881+0000","last_clean":"2024-05-03T17:53:18.872881+0000","last_became_active":"2024-05-03T17:52:35.735612+0000","last_became_peered":"2024-05-03T17:52:35.735612+0000","last_unstale":"2024-05-03T17:53:18.872881+0000","last_undegraded":"2024-05-03T17:53:18.872881+0000","last_fullsized":"2024-05-03T17:53:18.872881+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:59.216022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968246+0000","last_change":"2024-05-03T17:52:32.687147+0000","last_active":"2024-05-03T17:53:18.968246+0000","last_peered":"2024-05-03T17:53:18.968246+0000","last_clean":"2024-05-03T17:53:18.968246+0000","last_became_active":"2024-05-03T17:52:32.686831+0000","last_became_peered":"2024-05-03T17:52:32.686831+0000","last_unstale":"2024-05-03T17:53:18.968246+0000","last_undegraded":"2024-05-03T17:53:18.968246+0000","last_fullsized":"2024-05-03T17:53:18.968246+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:24:30.092829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.869761+0000","last_change":"2024-05-03T17:52:35.721862+0000","last_active":"2024-05-03T17:53:18.869761+0000","last_peered":"2024-05-03T17:53:18.869761+0000","last_clean":"2024-05-03T17:53:18.869761+0000","last_became_active":"2024-05-03T17:52:35.721393+0000","last_became_peered":"2024-05-03T17:52:35.721393+0000","last_unstale":"2024-05-03T17:53:18.869761+0000","last_undegraded":"2024-05-03T17:53:18.869761+0000","last_fullsized":"2024-05-03T17:53:18.869761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:47:32.014817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885948+0000","last_change":"2024-05-03T17:52:32.687867+0000","last_active":"2024-05-03T17:53:19.885948+0000","last_peered":"2024-05-03T17:53:19.885948+0000","last_clean":"2024-05-03T17:53:19.885948+0000","last_became_active":"2024-05-03T17:52:32.687557+0000","last_became_peered":"2024-05-03T17:52:32.687557+0000","last_unstale":"2024-05-03T17:53:19.885948+0000","last_undegraded":"2024-05-03T17:53:19.885948+0000","last_fullsized":"2024-05-03T17:53:19.885948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:45.863180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968459+0000","last_change":"2024-05-03T17:52:35.731057+0000","last_active":"2024-05-03T17:53:18.968459+0000","last_peered":"2024-05-03T17:53:18.968459+0000","last_clean":"2024-05-03T17:53:18.968459+0000","last_became_active":"2024-05-03T17:52:35.729293+0000","last_became_peered":"2024-05-03T17:52:35.729293+0000","last_unstale":"2024-05-03T17:53:18.968459+0000","last_undegraded":"2024-05-03T17:53:18.968459+0000","last_fullsized":"2024-05-03T17:53:18.968459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:12:53.888078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.872459+0000","last_change":"2024-05-03T17:52:32.689675+0000","last_active":"2024-05-03T17:53:18.872459+0000","last_peered":"2024-05-03T17:53:18.872459+0000","last_clean":"2024-05-03T17:53:18.872459+0000","last_became_active":"2024-05-03T17:52:32.689340+0000","last_became_peered":"2024-05-03T17:52:32.689340+0000","last_unstale":"2024-05-03T17:53:18.872459+0000","last_undegraded":"2024-05-03T17:53:18.872459+0000","last_fullsized":"2024-05-03T17:53:18.872459+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:31.398045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715044+0000","last_change":"2024-05-03T17:52:35.728228+0000","last_active":"2024-05-03T17:52:37.715044+0000","last_peered":"2024-05-03T17:52:37.715044+0000","last_clean":"2024-05-03T17:52:37.715044+0000","last_became_active":"2024-05-03T17:52:35.727233+0000","last_became_peered":"2024-05-03T17:52:35.727233+0000","last_unstale":"2024-05-03T17:52:37.715044+0000","last_undegraded":"2024-05-03T17:52:37.715044+0000","last_fullsized":"2024-05-03T17:52:37.715044+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:57.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886416+0000","last_change":"2024-05-03T17:52:33.249690+0000","last_active":"2024-05-03T17:53:19.886416+0000","last_peered":"2024-05-03T17:53:19.886416+0000","last_clean":"2024-05-03T17:53:19.886416+0000","last_became_active":"2024-05-03T17:52:33.249302+0000","last_became_peered":"2024-05-03T17:52:33.249302+0000","last_unstale":"2024-05-03T17:53:19.886416+0000","last_undegraded":"2024-05-03T17:53:19.886416+0000","last_fullsized":"2024-05-03T17:53:19.886416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:05:48.700670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715448+0000","last_change":"2024-05-03T17:52:35.727171+0000","last_active":"2024-05-03T17:52:37.715448+0000","last_peered":"2024-05-03T17:52:37.715448+0000","last_clean":"2024-05-03T17:52:37.715448+0000","last_became_active":"2024-05-03T17:52:35.726834+0000","last_became_peered":"2024-05-03T17:52:35.726834+0000","last_unstale":"2024-05-03T17:52:37.715448+0000","last_undegraded":"2024-05-03T17:52:37.715448+0000","last_fullsized":"2024-05-03T17:52:37.715448+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:40:09.938183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873627+0000","last_change":"2024-05-03T17:52:32.693694+0000","last_active":"2024-05-03T17:53:18.873627+0000","last_peered":"2024-05-03T17:53:18.873627+0000","last_clean":"2024-05-03T17:53:18.873627+0000","last_became_active":"2024-05-03T17:52:32.693310+0000","last_became_peered":"2024-05-03T17:52:32.693310+0000","last_unstale":"2024-05-03T17:53:18.873627+0000","last_undegraded":"2024-05-03T17:53:18.873627+0000","last_fullsized":"2024-05-03T17:53:18.873627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:59:37.449434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:38.274848+0000","last_change":"2024-05-03T17:52:35.728267+0000","last_active":"2024-05-03T17:52:38.274848+0000","last_peered":"2024-05-03T17:52:38.274848+0000","last_clean":"2024-05-03T17:52:38.274848+0000","last_became_active":"2024-05-03T17:52:35.727902+0000","last_became_peered":"2024-05-03T17:52:35.727902+0000","last_unstale":"2024-05-03T17:52:38.274848+0000","last_undegraded":"2024-05-03T17:52:38.274848+0000","last_fullsized":"2024-05-03T17:52:38.274848+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:09.223850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.729054+0000","last_change":"2024-05-03T17:52:32.687248+0000","last_active":"2024-05-03T17:52:37.729054+0000","last_peered":"2024-05-03T17:52:37.729054+0000","last_clean":"2024-05-03T17:52:37.729054+0000","last_became_active":"2024-05-03T17:52:32.686921+0000","last_became_peered":"2024-05-03T17:52:32.686921+0000","last_unstale":"2024-05-03T17:52:37.729054+0000","last_undegraded":"2024-05-03T17:52:37.729054+0000","last_fullsized":"2024-05-03T17:52:37.729054+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:35:40.849484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.715402+0000","last_change":"2024-05-03T17:52:35.729383+0000","last_active":"2024-05-03T17:52:37.715402+0000","last_peered":"2024-05-03T17:52:37.715402+0000","last_clean":"2024-05-03T17:52:37.715402+0000","last_became_active":"2024-05-03T17:52:35.729037+0000","last_became_peered":"2024-05-03T17:52:35.729037+0000","last_unstale":"2024-05-03T17:52:37.715402+0000","last_undegraded":"2024-05-03T17:52:37.715402+0000","last_fullsized":"2024-05-03T17:52:37.715402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:03:03.815770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.869894+0000","last_change":"2024-05-03T17:52:35.731809+0000","last_active":"2024-05-03T17:53:18.869894+0000","last_peered":"2024-05-03T17:53:18.869894+0000","last_clean":"2024-05-03T17:53:18.869894+0000","last_became_active":"2024-05-03T17:52:35.729360+0000","last_became_peered":"2024-05-03T17:52:35.729360+0000","last_unstale":"2024-05-03T17:53:18.869894+0000","last_undegraded":"2024-05-03T17:53:18.869894+0000","last_fullsized":"2024-05-03T17:53:18.869894+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:22.181184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886168+0000","last_change":"2024-05-03T17:52:35.727771+0000","last_active":"2024-05-03T17:53:19.886168+0000","last_peered":"2024-05-03T17:53:19.886168+0000","last_clean":"2024-05-03T17:53:19.886168+0000","last_became_active":"2024-05-03T17:52:35.727414+0000","last_became_peered":"2024-05-03T17:52:35.727414+0000","last_unstale":"2024-05-03T17:53:19.886168+0000","last_undegraded":"2024-05-03T17:53:19.886168+0000","last_fullsized":"2024-05-03T17:53:19.886168+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:09:02.905481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885867+0000","last_change":"2024-05-03T17:52:35.727433+0000","last_active":"2024-05-03T17:53:19.885867+0000","last_peered":"2024-05-03T17:53:19.885867+0000","last_clean":"2024-05-03T17:53:19.885867+0000","last_became_active":"2024-05-03T17:52:35.727123+0000","last_became_peered":"2024-05-03T17:52:35.727123+0000","last_unstale":"2024-05-03T17:53:19.885867+0000","last_undegraded":"2024-05-03T17:53:19.885867+0000","last_fullsized":"2024-05-03T17:53:19.885867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:41:36.871788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.722924+0000","last_change":"2024-05-03T17:52:35.721332+0000","last_active":"2024-05-03T17:52:37.722924+0000","last_peered":"2024-05-03T17:52:37.722924+0000","last_clean":"2024-05-03T17:52:37.722924+0000","last_became_active":"2024-05-03T17:52:35.721004+0000","last_became_peered":"2024-05-03T17:52:35.721004+0000","last_unstale":"2024-05-03T17:52:37.722924+0000","last_undegraded":"2024-05-03T17:52:37.722924+0000","last_fullsized":"2024-05-03T17:52:37.722924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:35:03.443163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721268+0000","last_change":"2024-05-03T17:52:35.723649+0000","last_active":"2024-05-03T17:52:37.721268+0000","last_peered":"2024-05-03T17:52:37.721268+0000","last_clean":"2024-05-03T17:52:37.721268+0000","last_became_active":"2024-05-03T17:52:35.723303+0000","last_became_peered":"2024-05-03T17:52:35.723303+0000","last_unstale":"2024-05-03T17:52:37.721268+0000","last_undegraded":"2024-05-03T17:52:37.721268+0000","last_fullsized":"2024-05-03T17:52:37.721268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:02:49.093039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.870949+0000","last_change":"2024-05-03T17:52:35.732505+0000","last_active":"2024-05-03T17:53:18.870949+0000","last_peered":"2024-05-03T17:53:18.870949+0000","last_clean":"2024-05-03T17:53:18.870949+0000","last_became_active":"2024-05-03T17:52:35.732180+0000","last_became_peered":"2024-05-03T17:52:35.732180+0000","last_unstale":"2024-05-03T17:53:18.870949+0000","last_undegraded":"2024-05-03T17:53:18.870949+0000","last_fullsized":"2024-05-03T17:53:18.870949+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.371260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886349+0000","last_change":"2024-05-03T17:52:35.732217+0000","last_active":"2024-05-03T17:53:19.886349+0000","last_peered":"2024-05-03T17:53:19.886349+0000","last_clean":"2024-05-03T17:53:19.886349+0000","last_became_active":"2024-05-03T17:52:35.731868+0000","last_became_peered":"2024-05-03T17:52:35.731868+0000","last_unstale":"2024-05-03T17:53:19.886349+0000","last_undegraded":"2024-05-03T17:53:19.886349+0000","last_fullsized":"2024-05-03T17:53:19.886349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:04:17.521528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968577+0000","last_change":"2024-05-03T17:52:35.723641+0000","last_active":"2024-05-03T17:53:18.968577+0000","last_peered":"2024-05-03T17:53:18.968577+0000","last_clean":"2024-05-03T17:53:18.968577+0000","last_became_active":"2024-05-03T17:52:35.723323+0000","last_became_peered":"2024-05-03T17:52:35.723323+0000","last_unstale":"2024-05-03T17:53:18.968577+0000","last_undegraded":"2024-05-03T17:53:18.968577+0000","last_fullsized":"2024-05-03T17:53:18.968577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:45:34.032407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.873223+0000","last_change":"2024-05-03T17:52:35.731008+0000","last_active":"2024-05-03T17:53:18.873223+0000","last_peered":"2024-05-03T17:53:18.873223+0000","last_clean":"2024-05-03T17:53:18.873223+0000","last_became_active":"2024-05-03T17:52:35.730627+0000","last_became_peered":"2024-05-03T17:52:35.730627+0000","last_unstale":"2024-05-03T17:53:18.873223+0000","last_undegraded":"2024-05-03T17:53:18.873223+0000","last_fullsized":"2024-05-03T17:53:18.873223+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:30:32.884518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.968191+0000","last_change":"2024-05-03T17:52:35.726938+0000","last_active":"2024-05-03T17:53:18.968191+0000","last_peered":"2024-05-03T17:53:18.968191+0000","last_clean":"2024-05-03T17:53:18.968191+0000","last_became_active":"2024-05-03T17:52:35.726611+0000","last_became_peered":"2024-05-03T17:52:35.726611+0000","last_unstale":"2024-05-03T17:53:18.968191+0000","last_undegraded":"2024-05-03T17:53:18.968191+0000","last_fullsized":"2024-05-03T17:53:18.968191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:11.587035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867632+0000","last_change":"2024-05-03T17:52:35.722562+0000","last_active":"2024-05-03T17:53:19.867632+0000","last_peered":"2024-05-03T17:53:19.867632+0000","last_clean":"2024-05-03T17:53:19.867632+0000","last_became_active":"2024-05-03T17:52:35.722229+0000","last_became_peered":"2024-05-03T17:52:35.722229+0000","last_unstale":"2024-05-03T17:53:19.867632+0000","last_undegraded":"2024-05-03T17:53:19.867632+0000","last_fullsized":"2024-05-03T17:53:19.867632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:57:06.730533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-03T17:52:37.721608+0000","last_change":"2024-05-03T17:52:35.722707+0000","last_active":"2024-05-03T17:52:37.721608+0000","last_peered":"2024-05-03T17:52:37.721608+0000","last_clean":"2024-05-03T17:52:37.721608+0000","last_became_active":"2024-05-03T17:52:35.721847+0000","last_became_peered":"2024-05-03T17:52:35.721847+0000","last_unstale":"2024-05-03T17:52:37.721608+0000","last_undegraded":"2024-05-03T17:52:37.721608+0000","last_fullsized":"2024-05-03T17:52:37.721608+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:56:21.620074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T17:53:18.870385+0000","last_change":"2024-05-03T17:52:35.731810+0000","last_active":"2024-05-03T17:53:18.870385+0000","last_peered":"2024-05-03T17:53:18.870385+0000","last_clean":"2024-05-03T17:53:18.870385+0000","last_became_active":"2024-05-03T17:52:35.728986+0000","last_became_peered":"2024-05-03T17:52:35.728986+0000","last_unstale":"2024-05-03T17:53:18.870385+0000","last_undegraded":"2024-05-03T17:53:18.870385+0000","last_fullsized":"2024-05-03T17:53:18.870385+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:18.275792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280811,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57557,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":192,"num_read_kb":158,"num_write":160,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":169107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743449,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":839680,"data_stored":409579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1063009,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070500,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":418208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266728,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":843776,"data_stored":401208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":993529,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":831488,"data_stored":400677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":466991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":851968,"data_stored":401948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":460339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":860160,"data_stored":409515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":847872,"data_stored":410032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":992822,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":53165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":51342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T17:53:24.591 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T17:53:24.830 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:25.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:24 smithi122 ceph-mon[29744]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:25.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:24 smithi149 ceph-mon[33409]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:25.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:24 smithi192 ceph-mon[33753]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:25.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:25 smithi122 ceph-mon[29744]: pgmap v312: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:26.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:25 smithi149 ceph-mon[33409]: pgmap v312: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:26.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:25 smithi192 ceph-mon[33753]: pgmap v312: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:26.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.501+0000 7f0dca5bd700 1 -- 172.21.15.122:0/847900787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc406bb40 msgr2=0x7f0dc4110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:26.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.501+0000 7f0dca5bd700 1 --2- 172.21.15.122:0/847900787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc406bb40 0x7f0dc4110330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00b0a0 tx=0x7f0dbc00b3b0 comp rx=0 tx=0).stop 2024-05-03T17:53:26.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.501+0000 7f0dca5bd700 1 -- 172.21.15.122:0/847900787 shutdown_connections 2024-05-03T17:53:26.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.501+0000 7f0dca5bd700 1 --2- 172.21.15.122:0/847900787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc406bb40 0x7f0dc4110330 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:26.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.501+0000 7f0dca5bd700 1 --2- 172.21.15.122:0/847900787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc4109510 0x7f0dc41098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:26.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.501+0000 7f0dca5bd700 1 --2- 172.21.15.122:0/847900787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dc406b180 0x7f0dc406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:26.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.501+0000 7f0dca5bd700 1 -- 172.21.15.122:0/847900787 >> 172.21.15.122:0/847900787 conn(0x7f0dc4079350 msgr2=0x7f0dc4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:26.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.501+0000 7f0dca5bd700 1 -- 172.21.15.122:0/847900787 shutdown_connections 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.501+0000 7f0dca5bd700 1 -- 172.21.15.122:0/847900787 wait complete. 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dca5bd700 1 Processor -- start 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dca5bd700 1 -- start start 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dca5bd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dc406b180 0x7f0dc4102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dca5bd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc406bb40 0x7f0dc41029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dca5bd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc4109510 0x7f0dc4102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dca5bd700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc41136c0 con 0x7f0dc406bb40 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dca5bd700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4113540 con 0x7f0dc4109510 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dca5bd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4113840 con 0x7f0dc406b180 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc406bb40 0x7f0dc41029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc406bb40 0x7f0dc41029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39260/0 (socket says 172.21.15.122:39260) 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dc37fe700 1 -- 172.21.15.122:0/203343068 learned_addr learned my addr 172.21.15.122:0/203343068 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dc3fff700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dc406b180 0x7f0dc4102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:26.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dc8b5a700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc4109510 0x7f0dc4102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dc37fe700 1 -- 172.21.15.122:0/203343068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dc406b180 msgr2=0x7f0dc4102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dc37fe700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dc406b180 0x7f0dc4102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dc37fe700 1 -- 172.21.15.122:0/203343068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc4109510 msgr2=0x7f0dc4102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dc37fe700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc4109510 0x7f0dc4102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dc37fe700 1 -- 172.21.15.122:0/203343068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc41a80c0 con 0x7f0dc406bb40 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.502+0000 7f0dc3fff700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dc406b180 0x7f0dc4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.503+0000 7f0dc37fe700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc406bb40 0x7f0dc41029b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0db400c970 tx=0x7f0db400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.503+0000 7f0dc17fa700 1 -- 172.21.15.122:0/203343068 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4007aa0 con 0x7f0dc406bb40 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.503+0000 7f0dc17fa700 1 -- 172.21.15.122:0/203343068 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db4004ce0 con 0x7f0dc406bb40 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.503+0000 7f0dca5bd700 1 -- 172.21.15.122:0/203343068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc41a83b0 con 0x7f0dc406bb40 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.503+0000 7f0dc17fa700 1 -- 172.21.15.122:0/203343068 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4005510 con 0x7f0dc406bb40 2024-05-03T17:53:26.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.503+0000 7f0dca5bd700 1 -- 172.21.15.122:0/203343068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc41a88f0 con 0x7f0dc406bb40 2024-05-03T17:53:26.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.504+0000 7f0dca5bd700 1 -- 172.21.15.122:0/203343068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc410b9f0 con 0x7f0dc406bb40 2024-05-03T17:53:26.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.505+0000 7f0dc17fa700 1 -- 172.21.15.122:0/203343068 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f0db4007c40 con 0x7f0dc406bb40 2024-05-03T17:53:26.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.506+0000 7f0dc17fa700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0dac0b2990 0x7f0dac0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:26.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.506+0000 7f0dc3fff700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0dac0b2990 0x7f0dac0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:26.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.506+0000 7f0dc17fa700 1 -- 172.21.15.122:0/203343068 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f0db40e1fe0 con 0x7f0dc406bb40 2024-05-03T17:53:26.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.506+0000 7f0dc3fff700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0dac0b2990 0x7f0dac0b4e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0db80099d0 tx=0x7f0db8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:26.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.510+0000 7f0dc17fa700 1 -- 172.21.15.122:0/203343068 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0db40a9d30 con 0x7f0dc406bb40 2024-05-03T17:53:26.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.709+0000 7f0dca5bd700 1 -- 172.21.15.122:0/203343068 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0dc40638e0 con 0x7f0dac0b2990 2024-05-03T17:53:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.716+0000 7f0dc17fa700 1 -- 172.21.15.122:0/203343068 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258859 (secure 0 0 0) 0x7f0dc40638e0 con 0x7f0dac0b2990 2024-05-03T17:53:26.715 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:53:26.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.720+0000 7f0dca5bd700 1 -- 172.21.15.122:0/203343068 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0dac0b2990 msgr2=0x7f0dac0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:26.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.720+0000 7f0dca5bd700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0dac0b2990 0x7f0dac0b4e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0db80099d0 tx=0x7f0db8008040 comp rx=0 tx=0).stop 2024-05-03T17:53:26.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.720+0000 7f0dca5bd700 1 -- 172.21.15.122:0/203343068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc406bb40 msgr2=0x7f0dc41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:26.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.720+0000 7f0dca5bd700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc406bb40 0x7f0dc41029b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0db400c970 tx=0x7f0db400cc80 comp rx=0 tx=0).stop 2024-05-03T17:53:26.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.721+0000 7f0dca5bd700 1 -- 172.21.15.122:0/203343068 shutdown_connections 2024-05-03T17:53:26.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.721+0000 7f0dca5bd700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f0dac0b2990 0x7f0dac0b4e50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:26.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.721+0000 7f0dca5bd700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc4109510 0x7f0dc4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:26.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.721+0000 7f0dca5bd700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dc406b180 0x7f0dc4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:26.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.721+0000 7f0dca5bd700 1 --2- 172.21.15.122:0/203343068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc406bb40 0x7f0dc41029b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:26.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.721+0000 7f0dca5bd700 1 -- 172.21.15.122:0/203343068 >> 172.21.15.122:0/203343068 conn(0x7f0dc4079350 msgr2=0x7f0dc4100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:26.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.721+0000 7f0dca5bd700 1 -- 172.21.15.122:0/203343068 shutdown_connections 2024-05-03T17:53:26.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:26.721+0000 7f0dca5bd700 1 -- 172.21.15.122:0/203343068 wait complete. 2024-05-03T17:53:26.718 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-05-03T17:53:27.380 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-05-03T17:53:26.549840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648159,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":283,"num_read_kb":227,"num_write":285,"num_write_kb":2255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376332,"kb_used_data":10488,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565492,"statfs":{"total":1151940427776,"available":1151555063808,"internally_reserved":0,"allocated":10739712,"data_stored":6953157,"data_compressed":107130,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3564,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":19,"num_write":3,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001960"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870114+0000","last_change":"2024-05-03T17:52:35.725057+0000","last_active":"2024-05-03T17:53:19.870114+0000","last_peered":"2024-05-03T17:53:19.870114+0000","last_clean":"2024-05-03T17:53:19.870114+0000","last_became_active":"2024-05-03T17:52:35.724632+0000","last_became_peered":"2024-05-03T17:52:35.724632+0000","last_unstale":"2024-05-03T17:53:19.870114+0000","last_undegraded":"2024-05-03T17:53:19.870114+0000","last_fullsized":"2024-05-03T17:53:19.870114+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:46.065357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870650+0000","last_change":"2024-05-03T17:52:35.728500+0000","last_active":"2024-05-03T17:53:19.870650+0000","last_peered":"2024-05-03T17:53:19.870650+0000","last_clean":"2024-05-03T17:53:19.870650+0000","last_became_active":"2024-05-03T17:52:35.728210+0000","last_became_peered":"2024-05-03T17:52:35.728210+0000","last_unstale":"2024-05-03T17:53:19.870650+0000","last_undegraded":"2024-05-03T17:53:19.870650+0000","last_fullsized":"2024-05-03T17:53:19.870650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:33:23.410617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870644+0000","last_change":"2024-05-03T17:52:35.726300+0000","last_active":"2024-05-03T17:53:19.870644+0000","last_peered":"2024-05-03T17:53:19.870644+0000","last_clean":"2024-05-03T17:53:19.870644+0000","last_became_active":"2024-05-03T17:52:35.725994+0000","last_became_peered":"2024-05-03T17:52:35.725994+0000","last_unstale":"2024-05-03T17:53:19.870644+0000","last_undegraded":"2024-05-03T17:53:19.870644+0000","last_fullsized":"2024-05-03T17:53:19.870644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:31:19.723453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077278+0000","last_change":"2024-05-03T17:52:35.728105+0000","last_active":"2024-05-03T17:53:20.077278+0000","last_peered":"2024-05-03T17:53:20.077278+0000","last_clean":"2024-05-03T17:53:20.077278+0000","last_became_active":"2024-05-03T17:52:35.727762+0000","last_became_peered":"2024-05-03T17:52:35.727762+0000","last_unstale":"2024-05-03T17:53:20.077278+0000","last_undegraded":"2024-05-03T17:53:20.077278+0000","last_fullsized":"2024-05-03T17:53:20.077278+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.640057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870597+0000","last_change":"2024-05-03T17:52:35.734516+0000","last_active":"2024-05-03T17:53:19.870597+0000","last_peered":"2024-05-03T17:53:19.870597+0000","last_clean":"2024-05-03T17:53:19.870597+0000","last_became_active":"2024-05-03T17:52:35.734125+0000","last_became_peered":"2024-05-03T17:52:35.734125+0000","last_unstale":"2024-05-03T17:53:19.870597+0000","last_undegraded":"2024-05-03T17:53:19.870597+0000","last_fullsized":"2024-05-03T17:53:19.870597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:39:25.103137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871539+0000","last_change":"2024-05-03T17:52:35.723427+0000","last_active":"2024-05-03T17:53:19.871539+0000","last_peered":"2024-05-03T17:53:19.871539+0000","last_clean":"2024-05-03T17:53:19.871539+0000","last_became_active":"2024-05-03T17:52:35.723113+0000","last_became_peered":"2024-05-03T17:52:35.723113+0000","last_unstale":"2024-05-03T17:53:19.871539+0000","last_undegraded":"2024-05-03T17:53:19.871539+0000","last_fullsized":"2024-05-03T17:53:19.871539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:37:14.606347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077441+0000","last_change":"2024-05-03T17:52:35.730858+0000","last_active":"2024-05-03T17:53:20.077441+0000","last_peered":"2024-05-03T17:53:20.077441+0000","last_clean":"2024-05-03T17:53:20.077441+0000","last_became_active":"2024-05-03T17:52:35.730453+0000","last_became_peered":"2024-05-03T17:52:35.730453+0000","last_unstale":"2024-05-03T17:53:20.077441+0000","last_undegraded":"2024-05-03T17:53:20.077441+0000","last_fullsized":"2024-05-03T17:53:20.077441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:21:17.803027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076565+0000","last_change":"2024-05-03T17:52:35.730547+0000","last_active":"2024-05-03T17:53:20.076565+0000","last_peered":"2024-05-03T17:53:20.076565+0000","last_clean":"2024-05-03T17:53:20.076565+0000","last_became_active":"2024-05-03T17:52:35.729582+0000","last_became_peered":"2024-05-03T17:52:35.729582+0000","last_unstale":"2024-05-03T17:53:20.076565+0000","last_undegraded":"2024-05-03T17:53:20.076565+0000","last_fullsized":"2024-05-03T17:53:20.076565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:24:55.110537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.868010+0000","last_change":"2024-05-03T17:52:35.726305+0000","last_active":"2024-05-03T17:53:19.868010+0000","last_peered":"2024-05-03T17:53:19.868010+0000","last_clean":"2024-05-03T17:53:19.868010+0000","last_became_active":"2024-05-03T17:52:35.725858+0000","last_became_peered":"2024-05-03T17:52:35.725858+0000","last_unstale":"2024-05-03T17:53:19.868010+0000","last_undegraded":"2024-05-03T17:53:19.868010+0000","last_fullsized":"2024-05-03T17:53:19.868010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:59.347307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871846+0000","last_change":"2024-05-03T17:52:35.726865+0000","last_active":"2024-05-03T17:53:19.871846+0000","last_peered":"2024-05-03T17:53:19.871846+0000","last_clean":"2024-05-03T17:53:19.871846+0000","last_became_active":"2024-05-03T17:52:35.726483+0000","last_became_peered":"2024-05-03T17:52:35.726483+0000","last_unstale":"2024-05-03T17:53:19.871846+0000","last_undegraded":"2024-05-03T17:53:19.871846+0000","last_fullsized":"2024-05-03T17:53:19.871846+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:17.851133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077159+0000","last_change":"2024-05-03T17:52:35.730551+0000","last_active":"2024-05-03T17:53:20.077159+0000","last_peered":"2024-05-03T17:53:20.077159+0000","last_clean":"2024-05-03T17:53:20.077159+0000","last_became_active":"2024-05-03T17:52:35.730187+0000","last_became_peered":"2024-05-03T17:52:35.730187+0000","last_unstale":"2024-05-03T17:53:20.077159+0000","last_undegraded":"2024-05-03T17:53:20.077159+0000","last_fullsized":"2024-05-03T17:53:20.077159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:18:56.211061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077071+0000","last_change":"2024-05-03T17:52:35.730040+0000","last_active":"2024-05-03T17:53:20.077071+0000","last_peered":"2024-05-03T17:53:20.077071+0000","last_clean":"2024-05-03T17:53:20.077071+0000","last_became_active":"2024-05-03T17:52:35.729549+0000","last_became_peered":"2024-05-03T17:52:35.729549+0000","last_unstale":"2024-05-03T17:53:20.077071+0000","last_undegraded":"2024-05-03T17:53:20.077071+0000","last_fullsized":"2024-05-03T17:53:20.077071+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:50:05.052247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867289+0000","last_change":"2024-05-03T17:52:35.726709+0000","last_active":"2024-05-03T17:53:19.867289+0000","last_peered":"2024-05-03T17:53:19.867289+0000","last_clean":"2024-05-03T17:53:19.867289+0000","last_became_active":"2024-05-03T17:52:35.726166+0000","last_became_peered":"2024-05-03T17:52:35.726166+0000","last_unstale":"2024-05-03T17:53:19.867289+0000","last_undegraded":"2024-05-03T17:53:19.867289+0000","last_fullsized":"2024-05-03T17:53:19.867289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.164622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870439+0000","last_change":"2024-05-03T17:52:35.727583+0000","last_active":"2024-05-03T17:53:19.870439+0000","last_peered":"2024-05-03T17:53:19.870439+0000","last_clean":"2024-05-03T17:53:19.870439+0000","last_became_active":"2024-05-03T17:52:35.727131+0000","last_became_peered":"2024-05-03T17:52:35.727131+0000","last_unstale":"2024-05-03T17:53:19.870439+0000","last_undegraded":"2024-05-03T17:53:19.870439+0000","last_fullsized":"2024-05-03T17:53:19.870439+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:28:21.255954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.887563+0000","last_change":"2024-05-03T17:53:19.887563+0000","last_active":"2024-05-03T17:53:19.887563+0000","last_peered":"2024-05-03T17:53:19.887563+0000","last_clean":"2024-05-03T17:53:19.887563+0000","last_became_active":"2024-05-03T17:53:19.887068+0000","last_became_peered":"2024-05-03T17:53:19.887068+0000","last_unstale":"2024-05-03T17:53:19.887563+0000","last_undegraded":"2024-05-03T17:53:19.887563+0000","last_fullsized":"2024-05-03T17:53:19.887563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:49:49.354138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867360+0000","last_change":"2024-05-03T17:52:35.717354+0000","last_active":"2024-05-03T17:53:19.867360+0000","last_peered":"2024-05-03T17:53:19.867360+0000","last_clean":"2024-05-03T17:53:19.867360+0000","last_became_active":"2024-05-03T17:52:35.717025+0000","last_became_peered":"2024-05-03T17:52:35.717025+0000","last_unstale":"2024-05-03T17:53:19.867360+0000","last_undegraded":"2024-05-03T17:53:19.867360+0000","last_fullsized":"2024-05-03T17:53:19.867360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:11:07.778484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870387+0000","last_change":"2024-05-03T17:52:35.725626+0000","last_active":"2024-05-03T17:53:19.870387+0000","last_peered":"2024-05-03T17:53:19.870387+0000","last_clean":"2024-05-03T17:53:19.870387+0000","last_became_active":"2024-05-03T17:52:35.725239+0000","last_became_peered":"2024-05-03T17:52:35.725239+0000","last_unstale":"2024-05-03T17:53:19.870387+0000","last_undegraded":"2024-05-03T17:53:19.870387+0000","last_fullsized":"2024-05-03T17:53:19.870387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:44:18.359484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869622+0000","last_change":"2024-05-03T17:52:35.724250+0000","last_active":"2024-05-03T17:53:19.869622+0000","last_peered":"2024-05-03T17:53:19.869622+0000","last_clean":"2024-05-03T17:53:19.869622+0000","last_became_active":"2024-05-03T17:52:35.723998+0000","last_became_peered":"2024-05-03T17:52:35.723998+0000","last_unstale":"2024-05-03T17:53:19.869622+0000","last_undegraded":"2024-05-03T17:53:19.869622+0000","last_fullsized":"2024-05-03T17:53:19.869622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:03:32.939559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886262+0000","last_change":"2024-05-03T17:52:35.726237+0000","last_active":"2024-05-03T17:53:19.886262+0000","last_peered":"2024-05-03T17:53:19.886262+0000","last_clean":"2024-05-03T17:53:19.886262+0000","last_became_active":"2024-05-03T17:52:35.725847+0000","last_became_peered":"2024-05-03T17:52:35.725847+0000","last_unstale":"2024-05-03T17:53:19.886262+0000","last_undegraded":"2024-05-03T17:53:19.886262+0000","last_fullsized":"2024-05-03T17:53:19.886262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:06.472532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869723+0000","last_change":"2024-05-03T17:52:32.687198+0000","last_active":"2024-05-03T17:53:19.869723+0000","last_peered":"2024-05-03T17:53:19.869723+0000","last_clean":"2024-05-03T17:53:19.869723+0000","last_became_active":"2024-05-03T17:52:32.686848+0000","last_became_peered":"2024-05-03T17:52:32.686848+0000","last_unstale":"2024-05-03T17:53:19.869723+0000","last_undegraded":"2024-05-03T17:53:19.869723+0000","last_fullsized":"2024-05-03T17:53:19.869723+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:08:24.244371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076745+0000","last_change":"2024-05-03T17:52:35.730595+0000","last_active":"2024-05-03T17:53:20.076745+0000","last_peered":"2024-05-03T17:53:20.076745+0000","last_clean":"2024-05-03T17:53:20.076745+0000","last_became_active":"2024-05-03T17:52:35.729999+0000","last_became_peered":"2024-05-03T17:52:35.729999+0000","last_unstale":"2024-05-03T17:53:20.076745+0000","last_undegraded":"2024-05-03T17:53:20.076745+0000","last_fullsized":"2024-05-03T17:53:20.076745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:14.485756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076923+0000","last_change":"2024-05-03T17:52:32.935239+0000","last_active":"2024-05-03T17:53:20.076923+0000","last_peered":"2024-05-03T17:53:20.076923+0000","last_clean":"2024-05-03T17:53:20.076923+0000","last_became_active":"2024-05-03T17:52:32.934915+0000","last_became_peered":"2024-05-03T17:52:32.934915+0000","last_unstale":"2024-05-03T17:53:20.076923+0000","last_undegraded":"2024-05-03T17:53:20.076923+0000","last_fullsized":"2024-05-03T17:53:20.076923+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:20.785071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870934+0000","last_change":"2024-05-03T17:52:35.724940+0000","last_active":"2024-05-03T17:53:19.870934+0000","last_peered":"2024-05-03T17:53:19.870934+0000","last_clean":"2024-05-03T17:53:19.870934+0000","last_became_active":"2024-05-03T17:52:35.724603+0000","last_became_peered":"2024-05-03T17:52:35.724603+0000","last_unstale":"2024-05-03T17:53:19.870934+0000","last_undegraded":"2024-05-03T17:53:19.870934+0000","last_fullsized":"2024-05-03T17:53:19.870934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:43.640499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871451+0000","last_change":"2024-05-03T17:52:32.688202+0000","last_active":"2024-05-03T17:53:19.871451+0000","last_peered":"2024-05-03T17:53:19.871451+0000","last_clean":"2024-05-03T17:53:19.871451+0000","last_became_active":"2024-05-03T17:52:32.687893+0000","last_became_peered":"2024-05-03T17:52:32.687893+0000","last_unstale":"2024-05-03T17:53:19.871451+0000","last_undegraded":"2024-05-03T17:53:19.871451+0000","last_fullsized":"2024-05-03T17:53:19.871451+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:43.342832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867555+0000","last_change":"2024-05-03T17:52:35.720980+0000","last_active":"2024-05-03T17:53:19.867555+0000","last_peered":"2024-05-03T17:53:19.867555+0000","last_clean":"2024-05-03T17:53:19.867555+0000","last_became_active":"2024-05-03T17:52:35.720582+0000","last_became_peered":"2024-05-03T17:52:35.720582+0000","last_unstale":"2024-05-03T17:53:19.867555+0000","last_undegraded":"2024-05-03T17:53:19.867555+0000","last_fullsized":"2024-05-03T17:53:19.867555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:01:22.938186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.877543+0000","last_change":"2024-05-03T17:53:19.877543+0000","last_active":"2024-05-03T17:53:19.877543+0000","last_peered":"2024-05-03T17:53:19.877543+0000","last_clean":"2024-05-03T17:53:19.877543+0000","last_became_active":"2024-05-03T17:53:19.873988+0000","last_became_peered":"2024-05-03T17:53:19.873988+0000","last_unstale":"2024-05-03T17:53:19.877543+0000","last_undegraded":"2024-05-03T17:53:19.877543+0000","last_fullsized":"2024-05-03T17:53:19.877543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:49:49.105623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.079283+0000","last_change":"2024-05-03T17:53:20.078906+0000","last_active":"2024-05-03T17:53:20.079283+0000","last_peered":"2024-05-03T17:53:20.079283+0000","last_clean":"2024-05-03T17:53:20.079283+0000","last_became_active":"2024-05-03T17:53:20.078538+0000","last_became_peered":"2024-05-03T17:53:20.078538+0000","last_unstale":"2024-05-03T17:53:20.079283+0000","last_undegraded":"2024-05-03T17:53:20.079283+0000","last_fullsized":"2024-05-03T17:53:20.079283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:59:46.945417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:22.809725+0000","last_change":"2024-05-03T17:52:32.692735+0000","last_active":"2024-05-03T17:53:22.809725+0000","last_peered":"2024-05-03T17:53:22.809725+0000","last_clean":"2024-05-03T17:53:22.809725+0000","last_became_active":"2024-05-03T17:52:32.692394+0000","last_became_peered":"2024-05-03T17:52:32.692394+0000","last_unstale":"2024-05-03T17:53:22.809725+0000","last_undegraded":"2024-05-03T17:53:22.809725+0000","last_fullsized":"2024-05-03T17:53:22.809725+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:40:11.434751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870798+0000","last_change":"2024-05-03T17:52:35.727167+0000","last_active":"2024-05-03T17:53:19.870798+0000","last_peered":"2024-05-03T17:53:19.870798+0000","last_clean":"2024-05-03T17:53:19.870798+0000","last_became_active":"2024-05-03T17:52:35.726792+0000","last_became_peered":"2024-05-03T17:52:35.726792+0000","last_unstale":"2024-05-03T17:53:19.870798+0000","last_undegraded":"2024-05-03T17:53:19.870798+0000","last_fullsized":"2024-05-03T17:53:19.870798+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:54:01.823025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871452+0000","last_change":"2024-05-03T17:52:32.687573+0000","last_active":"2024-05-03T17:53:19.871452+0000","last_peered":"2024-05-03T17:53:19.871452+0000","last_clean":"2024-05-03T17:53:19.871452+0000","last_became_active":"2024-05-03T17:52:32.686158+0000","last_became_peered":"2024-05-03T17:52:32.686158+0000","last_unstale":"2024-05-03T17:53:19.871452+0000","last_undegraded":"2024-05-03T17:53:19.871452+0000","last_fullsized":"2024-05-03T17:53:19.871452+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:19.891479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873172+0000","last_change":"2024-05-03T17:52:35.728049+0000","last_active":"2024-05-03T17:53:19.873172+0000","last_peered":"2024-05-03T17:53:19.873172+0000","last_clean":"2024-05-03T17:53:19.873172+0000","last_became_active":"2024-05-03T17:52:35.727713+0000","last_became_peered":"2024-05-03T17:52:35.727713+0000","last_unstale":"2024-05-03T17:53:19.873172+0000","last_undegraded":"2024-05-03T17:53:19.873172+0000","last_fullsized":"2024-05-03T17:53:19.873172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:28:01.798457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867118+0000","last_change":"2024-05-03T17:52:32.685679+0000","last_active":"2024-05-03T17:53:19.867118+0000","last_peered":"2024-05-03T17:53:19.867118+0000","last_clean":"2024-05-03T17:53:19.867118+0000","last_became_active":"2024-05-03T17:52:32.685306+0000","last_became_peered":"2024-05-03T17:52:32.685306+0000","last_unstale":"2024-05-03T17:53:19.867118+0000","last_undegraded":"2024-05-03T17:53:19.867118+0000","last_fullsized":"2024-05-03T17:53:19.867118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:48:22.218310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076501+0000","last_change":"2024-05-03T17:52:35.717111+0000","last_active":"2024-05-03T17:53:20.076501+0000","last_peered":"2024-05-03T17:53:20.076501+0000","last_clean":"2024-05-03T17:53:20.076501+0000","last_became_active":"2024-05-03T17:52:35.716782+0000","last_became_peered":"2024-05-03T17:52:35.716782+0000","last_unstale":"2024-05-03T17:53:20.076501+0000","last_undegraded":"2024-05-03T17:53:20.076501+0000","last_fullsized":"2024-05-03T17:53:20.076501+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:31:57.766927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872477+0000","last_change":"2024-05-03T17:52:32.690846+0000","last_active":"2024-05-03T17:53:19.872477+0000","last_peered":"2024-05-03T17:53:19.872477+0000","last_clean":"2024-05-03T17:53:19.872477+0000","last_became_active":"2024-05-03T17:52:32.690506+0000","last_became_peered":"2024-05-03T17:52:32.690506+0000","last_unstale":"2024-05-03T17:53:19.872477+0000","last_undegraded":"2024-05-03T17:53:19.872477+0000","last_fullsized":"2024-05-03T17:53:19.872477+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:19.810205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869511+0000","last_change":"2024-05-03T17:52:35.718229+0000","last_active":"2024-05-03T17:53:19.869511+0000","last_peered":"2024-05-03T17:53:19.869511+0000","last_clean":"2024-05-03T17:53:19.869511+0000","last_became_active":"2024-05-03T17:52:35.717907+0000","last_became_peered":"2024-05-03T17:52:35.717907+0000","last_unstale":"2024-05-03T17:53:19.869511+0000","last_undegraded":"2024-05-03T17:53:19.869511+0000","last_fullsized":"2024-05-03T17:53:19.869511+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:45.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":58,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.805562+0000","last_change":"2024-05-03T17:52:32.691174+0000","last_active":"2024-05-03T17:53:20.805562+0000","last_peered":"2024-05-03T17:53:20.805562+0000","last_clean":"2024-05-03T17:53:20.805562+0000","last_became_active":"2024-05-03T17:52:32.689907+0000","last_became_peered":"2024-05-03T17:52:32.689907+0000","last_unstale":"2024-05-03T17:53:20.805562+0000","last_undegraded":"2024-05-03T17:53:20.805562+0000","last_fullsized":"2024-05-03T17:53:20.805562+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:46.630436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867946+0000","last_change":"2024-05-03T17:52:35.720290+0000","last_active":"2024-05-03T17:53:19.867946+0000","last_peered":"2024-05-03T17:53:19.867946+0000","last_clean":"2024-05-03T17:53:19.867946+0000","last_became_active":"2024-05-03T17:52:35.719962+0000","last_became_peered":"2024-05-03T17:52:35.719962+0000","last_unstale":"2024-05-03T17:53:19.867946+0000","last_undegraded":"2024-05-03T17:53:19.867946+0000","last_fullsized":"2024-05-03T17:53:19.867946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:31:41.301592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077710+0000","last_change":"2024-05-03T17:52:32.689937+0000","last_active":"2024-05-03T17:53:20.077710+0000","last_peered":"2024-05-03T17:53:20.077710+0000","last_clean":"2024-05-03T17:53:20.077710+0000","last_became_active":"2024-05-03T17:52:32.689624+0000","last_became_peered":"2024-05-03T17:52:32.689624+0000","last_unstale":"2024-05-03T17:53:20.077710+0000","last_undegraded":"2024-05-03T17:53:20.077710+0000","last_fullsized":"2024-05-03T17:53:20.077710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:24.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870881+0000","last_change":"2024-05-03T17:52:35.726824+0000","last_active":"2024-05-03T17:53:19.870881+0000","last_peered":"2024-05-03T17:53:19.870881+0000","last_clean":"2024-05-03T17:53:19.870881+0000","last_became_active":"2024-05-03T17:52:35.726494+0000","last_became_peered":"2024-05-03T17:52:35.726494+0000","last_unstale":"2024-05-03T17:53:19.870881+0000","last_undegraded":"2024-05-03T17:53:19.870881+0000","last_fullsized":"2024-05-03T17:53:19.870881+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:41:21.284852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873051+0000","last_change":"2024-05-03T17:52:32.935471+0000","last_active":"2024-05-03T17:53:19.873051+0000","last_peered":"2024-05-03T17:53:19.873051+0000","last_clean":"2024-05-03T17:53:19.873051+0000","last_became_active":"2024-05-03T17:52:32.935012+0000","last_became_peered":"2024-05-03T17:52:32.935012+0000","last_unstale":"2024-05-03T17:53:19.873051+0000","last_undegraded":"2024-05-03T17:53:19.873051+0000","last_fullsized":"2024-05-03T17:53:19.873051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:35:38.654577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867634+0000","last_change":"2024-05-03T17:52:35.727124+0000","last_active":"2024-05-03T17:53:19.867634+0000","last_peered":"2024-05-03T17:53:19.867634+0000","last_clean":"2024-05-03T17:53:19.867634+0000","last_became_active":"2024-05-03T17:52:35.726804+0000","last_became_peered":"2024-05-03T17:52:35.726804+0000","last_unstale":"2024-05-03T17:53:19.867634+0000","last_undegraded":"2024-05-03T17:53:19.867634+0000","last_fullsized":"2024-05-03T17:53:19.867634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:32.000803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'28","reported_seq":120,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:21.807474+0000","last_change":"2024-05-03T17:52:32.685014+0000","last_active":"2024-05-03T17:53:21.807474+0000","last_peered":"2024-05-03T17:53:21.807474+0000","last_clean":"2024-05-03T17:53:21.807474+0000","last_became_active":"2024-05-03T17:52:32.684660+0000","last_became_peered":"2024-05-03T17:52:32.684660+0000","last_unstale":"2024-05-03T17:53:21.807474+0000","last_undegraded":"2024-05-03T17:53:21.807474+0000","last_fullsized":"2024-05-03T17:53:21.807474+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:25:56.890283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53165,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":84,"num_write":29,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871281+0000","last_change":"2024-05-03T17:52:35.733675+0000","last_active":"2024-05-03T17:53:19.871281+0000","last_peered":"2024-05-03T17:53:19.871281+0000","last_clean":"2024-05-03T17:53:19.871281+0000","last_became_active":"2024-05-03T17:52:35.733234+0000","last_became_peered":"2024-05-03T17:52:35.733234+0000","last_unstale":"2024-05-03T17:53:19.871281+0000","last_undegraded":"2024-05-03T17:53:19.871281+0000","last_fullsized":"2024-05-03T17:53:19.871281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:57.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886182+0000","last_change":"2024-05-03T17:52:32.690462+0000","last_active":"2024-05-03T17:53:19.886182+0000","last_peered":"2024-05-03T17:53:19.886182+0000","last_clean":"2024-05-03T17:53:19.886182+0000","last_became_active":"2024-05-03T17:52:32.690169+0000","last_became_peered":"2024-05-03T17:52:32.690169+0000","last_unstale":"2024-05-03T17:53:19.886182+0000","last_undegraded":"2024-05-03T17:53:19.886182+0000","last_fullsized":"2024-05-03T17:53:19.886182+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:59.324612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867010+0000","last_change":"2024-05-03T17:52:35.720730+0000","last_active":"2024-05-03T17:53:19.867010+0000","last_peered":"2024-05-03T17:53:19.867010+0000","last_clean":"2024-05-03T17:53:19.867010+0000","last_became_active":"2024-05-03T17:52:35.720392+0000","last_became_peered":"2024-05-03T17:52:35.720392+0000","last_unstale":"2024-05-03T17:53:19.867010+0000","last_undegraded":"2024-05-03T17:53:19.867010+0000","last_fullsized":"2024-05-03T17:53:19.867010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:58.528148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872742+0000","last_change":"2024-05-03T17:52:32.691439+0000","last_active":"2024-05-03T17:53:19.872742+0000","last_peered":"2024-05-03T17:53:19.872742+0000","last_clean":"2024-05-03T17:53:19.872742+0000","last_became_active":"2024-05-03T17:52:32.691080+0000","last_became_peered":"2024-05-03T17:52:32.691080+0000","last_unstale":"2024-05-03T17:53:19.872742+0000","last_undegraded":"2024-05-03T17:53:19.872742+0000","last_fullsized":"2024-05-03T17:53:19.872742+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:40:43.120194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077160+0000","last_change":"2024-05-03T17:52:35.729860+0000","last_active":"2024-05-03T17:53:20.077160+0000","last_peered":"2024-05-03T17:53:20.077160+0000","last_clean":"2024-05-03T17:53:20.077160+0000","last_became_active":"2024-05-03T17:52:35.729508+0000","last_became_peered":"2024-05-03T17:52:35.729508+0000","last_unstale":"2024-05-03T17:53:20.077160+0000","last_undegraded":"2024-05-03T17:53:20.077160+0000","last_fullsized":"2024-05-03T17:53:20.077160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:39:20.859215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:22.809068+0000","last_change":"2024-05-03T17:52:32.689346+0000","last_active":"2024-05-03T17:53:22.809068+0000","last_peered":"2024-05-03T17:53:22.809068+0000","last_clean":"2024-05-03T17:53:22.809068+0000","last_became_active":"2024-05-03T17:52:32.689009+0000","last_became_peered":"2024-05-03T17:52:32.689009+0000","last_unstale":"2024-05-03T17:53:22.809068+0000","last_undegraded":"2024-05-03T17:53:22.809068+0000","last_fullsized":"2024-05-03T17:53:22.809068+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:55:36.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077075+0000","last_change":"2024-05-03T17:52:35.729226+0000","last_active":"2024-05-03T17:53:20.077075+0000","last_peered":"2024-05-03T17:53:20.077075+0000","last_clean":"2024-05-03T17:53:20.077075+0000","last_became_active":"2024-05-03T17:52:35.728846+0000","last_became_peered":"2024-05-03T17:52:35.728846+0000","last_unstale":"2024-05-03T17:53:20.077075+0000","last_undegraded":"2024-05-03T17:53:20.077075+0000","last_fullsized":"2024-05-03T17:53:20.077075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:39:16.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870410+0000","last_change":"2024-05-03T17:52:32.936108+0000","last_active":"2024-05-03T17:53:19.870410+0000","last_peered":"2024-05-03T17:53:19.870410+0000","last_clean":"2024-05-03T17:53:19.870410+0000","last_became_active":"2024-05-03T17:52:32.935782+0000","last_became_peered":"2024-05-03T17:52:32.935782+0000","last_unstale":"2024-05-03T17:53:19.870410+0000","last_undegraded":"2024-05-03T17:53:19.870410+0000","last_fullsized":"2024-05-03T17:53:19.870410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:47.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872695+0000","last_change":"2024-05-03T17:52:35.727163+0000","last_active":"2024-05-03T17:53:19.872695+0000","last_peered":"2024-05-03T17:53:19.872695+0000","last_clean":"2024-05-03T17:53:19.872695+0000","last_became_active":"2024-05-03T17:52:35.726824+0000","last_became_peered":"2024-05-03T17:52:35.726824+0000","last_unstale":"2024-05-03T17:53:19.872695+0000","last_undegraded":"2024-05-03T17:53:19.872695+0000","last_fullsized":"2024-05-03T17:53:19.872695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:24:50.803860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871546+0000","last_change":"2024-05-03T17:52:32.687329+0000","last_active":"2024-05-03T17:53:19.871546+0000","last_peered":"2024-05-03T17:53:19.871546+0000","last_clean":"2024-05-03T17:53:19.871546+0000","last_became_active":"2024-05-03T17:52:32.687025+0000","last_became_peered":"2024-05-03T17:52:32.687025+0000","last_unstale":"2024-05-03T17:53:19.871546+0000","last_undegraded":"2024-05-03T17:53:19.871546+0000","last_fullsized":"2024-05-03T17:53:19.871546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:42:15.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869955+0000","last_change":"2024-05-03T17:52:35.728797+0000","last_active":"2024-05-03T17:53:19.869955+0000","last_peered":"2024-05-03T17:53:19.869955+0000","last_clean":"2024-05-03T17:53:19.869955+0000","last_became_active":"2024-05-03T17:52:35.728429+0000","last_became_peered":"2024-05-03T17:52:35.728429+0000","last_unstale":"2024-05-03T17:53:19.869955+0000","last_undegraded":"2024-05-03T17:53:19.869955+0000","last_fullsized":"2024-05-03T17:53:19.869955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:19.584593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872508+0000","last_change":"2024-05-03T17:53:19.872508+0000","last_active":"2024-05-03T17:53:19.872508+0000","last_peered":"2024-05-03T17:53:19.872508+0000","last_clean":"2024-05-03T17:53:19.872508+0000","last_became_active":"2024-05-03T17:53:19.872175+0000","last_became_peered":"2024-05-03T17:53:19.872175+0000","last_unstale":"2024-05-03T17:53:19.872508+0000","last_undegraded":"2024-05-03T17:53:19.872508+0000","last_fullsized":"2024-05-03T17:53:19.872508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:24:08.246884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871588+0000","last_change":"2024-05-03T17:52:35.719115+0000","last_active":"2024-05-03T17:53:19.871588+0000","last_peered":"2024-05-03T17:53:19.871588+0000","last_clean":"2024-05-03T17:53:19.871588+0000","last_became_active":"2024-05-03T17:52:35.718054+0000","last_became_peered":"2024-05-03T17:52:35.718054+0000","last_unstale":"2024-05-03T17:53:19.871588+0000","last_undegraded":"2024-05-03T17:53:19.871588+0000","last_fullsized":"2024-05-03T17:53:19.871588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.903766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077572+0000","last_change":"2024-05-03T17:52:32.688993+0000","last_active":"2024-05-03T17:53:20.077572+0000","last_peered":"2024-05-03T17:53:20.077572+0000","last_clean":"2024-05-03T17:53:20.077572+0000","last_became_active":"2024-05-03T17:52:32.688706+0000","last_became_peered":"2024-05-03T17:52:32.688706+0000","last_unstale":"2024-05-03T17:53:20.077572+0000","last_undegraded":"2024-05-03T17:53:20.077572+0000","last_fullsized":"2024-05-03T17:53:20.077572+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:40:14.023338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872957+0000","last_change":"2024-05-03T17:53:19.872957+0000","last_active":"2024-05-03T17:53:19.872957+0000","last_peered":"2024-05-03T17:53:19.872957+0000","last_clean":"2024-05-03T17:53:19.872957+0000","last_became_active":"2024-05-03T17:53:19.872690+0000","last_became_peered":"2024-05-03T17:53:19.872690+0000","last_unstale":"2024-05-03T17:53:19.872957+0000","last_undegraded":"2024-05-03T17:53:19.872957+0000","last_fullsized":"2024-05-03T17:53:19.872957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:53:43.801983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872998+0000","last_change":"2024-05-03T17:52:35.728958+0000","last_active":"2024-05-03T17:53:19.872998+0000","last_peered":"2024-05-03T17:53:19.872998+0000","last_clean":"2024-05-03T17:53:19.872998+0000","last_became_active":"2024-05-03T17:52:35.728619+0000","last_became_peered":"2024-05-03T17:52:35.728619+0000","last_unstale":"2024-05-03T17:53:19.872998+0000","last_undegraded":"2024-05-03T17:53:19.872998+0000","last_fullsized":"2024-05-03T17:53:19.872998+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:29:29.922324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":60,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:21.806560+0000","last_change":"2024-05-03T17:52:32.935367+0000","last_active":"2024-05-03T17:53:21.806560+0000","last_peered":"2024-05-03T17:53:21.806560+0000","last_clean":"2024-05-03T17:53:21.806560+0000","last_became_active":"2024-05-03T17:52:32.934938+0000","last_became_peered":"2024-05-03T17:52:32.934938+0000","last_unstale":"2024-05-03T17:53:21.806560+0000","last_undegraded":"2024-05-03T17:53:21.806560+0000","last_fullsized":"2024-05-03T17:53:21.806560+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:07:21.642883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076663+0000","last_change":"2024-05-03T17:52:35.731035+0000","last_active":"2024-05-03T17:53:20.076663+0000","last_peered":"2024-05-03T17:53:20.076663+0000","last_clean":"2024-05-03T17:53:20.076663+0000","last_became_active":"2024-05-03T17:52:35.730034+0000","last_became_peered":"2024-05-03T17:52:35.730034+0000","last_unstale":"2024-05-03T17:53:20.076663+0000","last_undegraded":"2024-05-03T17:53:20.076663+0000","last_fullsized":"2024-05-03T17:53:20.076663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:25.545290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077849+0000","last_change":"2024-05-03T17:52:32.690043+0000","last_active":"2024-05-03T17:53:20.077849+0000","last_peered":"2024-05-03T17:53:20.077849+0000","last_clean":"2024-05-03T17:53:20.077849+0000","last_became_active":"2024-05-03T17:52:32.689562+0000","last_became_peered":"2024-05-03T17:52:32.689562+0000","last_unstale":"2024-05-03T17:53:20.077849+0000","last_undegraded":"2024-05-03T17:53:20.077849+0000","last_fullsized":"2024-05-03T17:53:20.077849+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:59.014968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":97,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077303+0000","last_change":"2024-05-03T17:51:02.340854+0000","last_active":"2024-05-03T17:53:20.077303+0000","last_peered":"2024-05-03T17:53:20.077303+0000","last_clean":"2024-05-03T17:53:20.077303+0000","last_became_active":"2024-05-03T17:51:02.336454+0000","last_became_peered":"2024-05-03T17:51:02.336454+0000","last_unstale":"2024-05-03T17:53:20.077303+0000","last_undegraded":"2024-05-03T17:53:20.077303+0000","last_fullsized":"2024-05-03T17:53:20.077303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_clean_scrub_stamp":"2024-05-03T17:47:41.306771+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:18:08.512098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.887126+0000","last_change":"2024-05-03T17:53:19.887126+0000","last_active":"2024-05-03T17:53:19.887126+0000","last_peered":"2024-05-03T17:53:19.887126+0000","last_clean":"2024-05-03T17:53:19.887126+0000","last_became_active":"2024-05-03T17:53:19.886736+0000","last_became_peered":"2024-05-03T17:53:19.886736+0000","last_unstale":"2024-05-03T17:53:19.887126+0000","last_undegraded":"2024-05-03T17:53:19.887126+0000","last_fullsized":"2024-05-03T17:53:19.887126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:35:02.036825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077402+0000","last_change":"2024-05-03T17:52:32.690093+0000","last_active":"2024-05-03T17:53:20.077402+0000","last_peered":"2024-05-03T17:53:20.077402+0000","last_clean":"2024-05-03T17:53:20.077402+0000","last_became_active":"2024-05-03T17:52:32.689777+0000","last_became_peered":"2024-05-03T17:52:32.689777+0000","last_unstale":"2024-05-03T17:53:20.077402+0000","last_undegraded":"2024-05-03T17:53:20.077402+0000","last_fullsized":"2024-05-03T17:53:20.077402+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:09.693011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871341+0000","last_change":"2024-05-03T17:52:32.935589+0000","last_active":"2024-05-03T17:53:19.871341+0000","last_peered":"2024-05-03T17:53:19.871341+0000","last_clean":"2024-05-03T17:53:19.871341+0000","last_became_active":"2024-05-03T17:52:32.935221+0000","last_became_peered":"2024-05-03T17:52:32.935221+0000","last_unstale":"2024-05-03T17:53:19.871341+0000","last_undegraded":"2024-05-03T17:53:19.871341+0000","last_fullsized":"2024-05-03T17:53:19.871341+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:26:08.001761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867191+0000","last_change":"2024-05-03T17:52:35.724293+0000","last_active":"2024-05-03T17:53:19.867191+0000","last_peered":"2024-05-03T17:53:19.867191+0000","last_clean":"2024-05-03T17:53:19.867191+0000","last_became_active":"2024-05-03T17:52:35.723978+0000","last_became_peered":"2024-05-03T17:52:35.723978+0000","last_unstale":"2024-05-03T17:53:19.867191+0000","last_undegraded":"2024-05-03T17:53:19.867191+0000","last_fullsized":"2024-05-03T17:53:19.867191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:23.357872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869626+0000","last_change":"2024-05-03T17:52:32.687303+0000","last_active":"2024-05-03T17:53:19.869626+0000","last_peered":"2024-05-03T17:53:19.869626+0000","last_clean":"2024-05-03T17:53:19.869626+0000","last_became_active":"2024-05-03T17:52:32.686827+0000","last_became_peered":"2024-05-03T17:52:32.686827+0000","last_unstale":"2024-05-03T17:53:19.869626+0000","last_undegraded":"2024-05-03T17:53:19.869626+0000","last_fullsized":"2024-05-03T17:53:19.869626+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:11:53.565215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076856+0000","last_change":"2024-05-03T17:52:35.720898+0000","last_active":"2024-05-03T17:53:20.076856+0000","last_peered":"2024-05-03T17:53:20.076856+0000","last_clean":"2024-05-03T17:53:20.076856+0000","last_became_active":"2024-05-03T17:52:35.720572+0000","last_became_peered":"2024-05-03T17:52:35.720572+0000","last_unstale":"2024-05-03T17:53:20.076856+0000","last_undegraded":"2024-05-03T17:53:20.076856+0000","last_fullsized":"2024-05-03T17:53:20.076856+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:01.697167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077357+0000","last_change":"2024-05-03T17:52:33.249970+0000","last_active":"2024-05-03T17:53:20.077357+0000","last_peered":"2024-05-03T17:53:20.077357+0000","last_clean":"2024-05-03T17:53:20.077357+0000","last_became_active":"2024-05-03T17:52:33.249596+0000","last_became_peered":"2024-05-03T17:52:33.249596+0000","last_unstale":"2024-05-03T17:53:20.077357+0000","last_undegraded":"2024-05-03T17:53:20.077357+0000","last_fullsized":"2024-05-03T17:53:20.077357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:46:53.058586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870807+0000","last_change":"2024-05-03T17:52:35.722129+0000","last_active":"2024-05-03T17:53:19.870807+0000","last_peered":"2024-05-03T17:53:19.870807+0000","last_clean":"2024-05-03T17:53:19.870807+0000","last_became_active":"2024-05-03T17:52:35.721815+0000","last_became_peered":"2024-05-03T17:52:35.721815+0000","last_unstale":"2024-05-03T17:53:19.870807+0000","last_undegraded":"2024-05-03T17:53:19.870807+0000","last_fullsized":"2024-05-03T17:53:19.870807+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:33.781350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867376+0000","last_change":"2024-05-03T17:52:32.936157+0000","last_active":"2024-05-03T17:53:19.867376+0000","last_peered":"2024-05-03T17:53:19.867376+0000","last_clean":"2024-05-03T17:53:19.867376+0000","last_became_active":"2024-05-03T17:52:32.935842+0000","last_became_peered":"2024-05-03T17:52:32.935842+0000","last_unstale":"2024-05-03T17:53:19.867376+0000","last_undegraded":"2024-05-03T17:53:19.867376+0000","last_fullsized":"2024-05-03T17:53:19.867376+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:18.006451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870494+0000","last_change":"2024-05-03T17:52:35.735993+0000","last_active":"2024-05-03T17:53:19.870494+0000","last_peered":"2024-05-03T17:53:19.870494+0000","last_clean":"2024-05-03T17:53:19.870494+0000","last_became_active":"2024-05-03T17:52:35.735612+0000","last_became_peered":"2024-05-03T17:52:35.735612+0000","last_unstale":"2024-05-03T17:53:19.870494+0000","last_undegraded":"2024-05-03T17:53:19.870494+0000","last_fullsized":"2024-05-03T17:53:19.870494+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:59.216022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076573+0000","last_change":"2024-05-03T17:52:32.687147+0000","last_active":"2024-05-03T17:53:20.076573+0000","last_peered":"2024-05-03T17:53:20.076573+0000","last_clean":"2024-05-03T17:53:20.076573+0000","last_became_active":"2024-05-03T17:52:32.686831+0000","last_became_peered":"2024-05-03T17:52:32.686831+0000","last_unstale":"2024-05-03T17:53:20.076573+0000","last_undegraded":"2024-05-03T17:53:20.076573+0000","last_fullsized":"2024-05-03T17:53:20.076573+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:24:30.092829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869678+0000","last_change":"2024-05-03T17:52:35.721862+0000","last_active":"2024-05-03T17:53:19.869678+0000","last_peered":"2024-05-03T17:53:19.869678+0000","last_clean":"2024-05-03T17:53:19.869678+0000","last_became_active":"2024-05-03T17:52:35.721393+0000","last_became_peered":"2024-05-03T17:52:35.721393+0000","last_unstale":"2024-05-03T17:53:19.869678+0000","last_undegraded":"2024-05-03T17:53:19.869678+0000","last_fullsized":"2024-05-03T17:53:19.869678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:47:32.014817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885948+0000","last_change":"2024-05-03T17:52:32.687867+0000","last_active":"2024-05-03T17:53:19.885948+0000","last_peered":"2024-05-03T17:53:19.885948+0000","last_clean":"2024-05-03T17:53:19.885948+0000","last_became_active":"2024-05-03T17:52:32.687557+0000","last_became_peered":"2024-05-03T17:52:32.687557+0000","last_unstale":"2024-05-03T17:53:19.885948+0000","last_undegraded":"2024-05-03T17:53:19.885948+0000","last_fullsized":"2024-05-03T17:53:19.885948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:45.863180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076840+0000","last_change":"2024-05-03T17:52:35.731057+0000","last_active":"2024-05-03T17:53:20.076840+0000","last_peered":"2024-05-03T17:53:20.076840+0000","last_clean":"2024-05-03T17:53:20.076840+0000","last_became_active":"2024-05-03T17:52:35.729293+0000","last_became_peered":"2024-05-03T17:52:35.729293+0000","last_unstale":"2024-05-03T17:53:20.076840+0000","last_undegraded":"2024-05-03T17:53:20.076840+0000","last_fullsized":"2024-05-03T17:53:20.076840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:12:53.888078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870721+0000","last_change":"2024-05-03T17:52:32.689675+0000","last_active":"2024-05-03T17:53:19.870721+0000","last_peered":"2024-05-03T17:53:19.870721+0000","last_clean":"2024-05-03T17:53:19.870721+0000","last_became_active":"2024-05-03T17:52:32.689340+0000","last_became_peered":"2024-05-03T17:52:32.689340+0000","last_unstale":"2024-05-03T17:53:19.870721+0000","last_undegraded":"2024-05-03T17:53:19.870721+0000","last_fullsized":"2024-05-03T17:53:19.870721+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:31.398045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077600+0000","last_change":"2024-05-03T17:52:35.728228+0000","last_active":"2024-05-03T17:53:20.077600+0000","last_peered":"2024-05-03T17:53:20.077600+0000","last_clean":"2024-05-03T17:53:20.077600+0000","last_became_active":"2024-05-03T17:52:35.727233+0000","last_became_peered":"2024-05-03T17:52:35.727233+0000","last_unstale":"2024-05-03T17:53:20.077600+0000","last_undegraded":"2024-05-03T17:53:20.077600+0000","last_fullsized":"2024-05-03T17:53:20.077600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:57.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886416+0000","last_change":"2024-05-03T17:52:33.249690+0000","last_active":"2024-05-03T17:53:19.886416+0000","last_peered":"2024-05-03T17:53:19.886416+0000","last_clean":"2024-05-03T17:53:19.886416+0000","last_became_active":"2024-05-03T17:52:33.249302+0000","last_became_peered":"2024-05-03T17:52:33.249302+0000","last_unstale":"2024-05-03T17:53:19.886416+0000","last_undegraded":"2024-05-03T17:53:19.886416+0000","last_fullsized":"2024-05-03T17:53:19.886416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:05:48.700670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077737+0000","last_change":"2024-05-03T17:52:35.727171+0000","last_active":"2024-05-03T17:53:20.077737+0000","last_peered":"2024-05-03T17:53:20.077737+0000","last_clean":"2024-05-03T17:53:20.077737+0000","last_became_active":"2024-05-03T17:52:35.726834+0000","last_became_peered":"2024-05-03T17:52:35.726834+0000","last_unstale":"2024-05-03T17:53:20.077737+0000","last_undegraded":"2024-05-03T17:53:20.077737+0000","last_fullsized":"2024-05-03T17:53:20.077737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:40:09.938183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872806+0000","last_change":"2024-05-03T17:52:32.693694+0000","last_active":"2024-05-03T17:53:19.872806+0000","last_peered":"2024-05-03T17:53:19.872806+0000","last_clean":"2024-05-03T17:53:19.872806+0000","last_became_active":"2024-05-03T17:52:32.693310+0000","last_became_peered":"2024-05-03T17:52:32.693310+0000","last_unstale":"2024-05-03T17:53:19.872806+0000","last_undegraded":"2024-05-03T17:53:19.872806+0000","last_fullsized":"2024-05-03T17:53:19.872806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:59:37.449434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077538+0000","last_change":"2024-05-03T17:52:35.728267+0000","last_active":"2024-05-03T17:53:20.077538+0000","last_peered":"2024-05-03T17:53:20.077538+0000","last_clean":"2024-05-03T17:53:20.077538+0000","last_became_active":"2024-05-03T17:52:35.727902+0000","last_became_peered":"2024-05-03T17:52:35.727902+0000","last_unstale":"2024-05-03T17:53:20.077538+0000","last_undegraded":"2024-05-03T17:53:20.077538+0000","last_fullsized":"2024-05-03T17:53:20.077538+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:09.223850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867146+0000","last_change":"2024-05-03T17:52:32.687248+0000","last_active":"2024-05-03T17:53:19.867146+0000","last_peered":"2024-05-03T17:53:19.867146+0000","last_clean":"2024-05-03T17:53:19.867146+0000","last_became_active":"2024-05-03T17:52:32.686921+0000","last_became_peered":"2024-05-03T17:52:32.686921+0000","last_unstale":"2024-05-03T17:53:19.867146+0000","last_undegraded":"2024-05-03T17:53:19.867146+0000","last_fullsized":"2024-05-03T17:53:19.867146+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:35:40.849484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077858+0000","last_change":"2024-05-03T17:52:35.729383+0000","last_active":"2024-05-03T17:53:20.077858+0000","last_peered":"2024-05-03T17:53:20.077858+0000","last_clean":"2024-05-03T17:53:20.077858+0000","last_became_active":"2024-05-03T17:52:35.729037+0000","last_became_peered":"2024-05-03T17:52:35.729037+0000","last_unstale":"2024-05-03T17:53:20.077858+0000","last_undegraded":"2024-05-03T17:53:20.077858+0000","last_fullsized":"2024-05-03T17:53:20.077858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:03:03.815770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869741+0000","last_change":"2024-05-03T17:52:35.731809+0000","last_active":"2024-05-03T17:53:19.869741+0000","last_peered":"2024-05-03T17:53:19.869741+0000","last_clean":"2024-05-03T17:53:19.869741+0000","last_became_active":"2024-05-03T17:52:35.729360+0000","last_became_peered":"2024-05-03T17:52:35.729360+0000","last_unstale":"2024-05-03T17:53:19.869741+0000","last_undegraded":"2024-05-03T17:53:19.869741+0000","last_fullsized":"2024-05-03T17:53:19.869741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:22.181184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886168+0000","last_change":"2024-05-03T17:52:35.727771+0000","last_active":"2024-05-03T17:53:19.886168+0000","last_peered":"2024-05-03T17:53:19.886168+0000","last_clean":"2024-05-03T17:53:19.886168+0000","last_became_active":"2024-05-03T17:52:35.727414+0000","last_became_peered":"2024-05-03T17:52:35.727414+0000","last_unstale":"2024-05-03T17:53:19.886168+0000","last_undegraded":"2024-05-03T17:53:19.886168+0000","last_fullsized":"2024-05-03T17:53:19.886168+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:09:02.905481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885867+0000","last_change":"2024-05-03T17:52:35.727433+0000","last_active":"2024-05-03T17:53:19.885867+0000","last_peered":"2024-05-03T17:53:19.885867+0000","last_clean":"2024-05-03T17:53:19.885867+0000","last_became_active":"2024-05-03T17:52:35.727123+0000","last_became_peered":"2024-05-03T17:52:35.727123+0000","last_unstale":"2024-05-03T17:53:19.885867+0000","last_undegraded":"2024-05-03T17:53:19.885867+0000","last_fullsized":"2024-05-03T17:53:19.885867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:41:36.871788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870666+0000","last_change":"2024-05-03T17:52:35.721332+0000","last_active":"2024-05-03T17:53:19.870666+0000","last_peered":"2024-05-03T17:53:19.870666+0000","last_clean":"2024-05-03T17:53:19.870666+0000","last_became_active":"2024-05-03T17:52:35.721004+0000","last_became_peered":"2024-05-03T17:52:35.721004+0000","last_unstale":"2024-05-03T17:53:19.870666+0000","last_undegraded":"2024-05-03T17:53:19.870666+0000","last_fullsized":"2024-05-03T17:53:19.870666+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:35:03.443163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867305+0000","last_change":"2024-05-03T17:52:35.723649+0000","last_active":"2024-05-03T17:53:19.867305+0000","last_peered":"2024-05-03T17:53:19.867305+0000","last_clean":"2024-05-03T17:53:19.867305+0000","last_became_active":"2024-05-03T17:52:35.723303+0000","last_became_peered":"2024-05-03T17:52:35.723303+0000","last_unstale":"2024-05-03T17:53:19.867305+0000","last_undegraded":"2024-05-03T17:53:19.867305+0000","last_fullsized":"2024-05-03T17:53:19.867305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:02:49.093039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870096+0000","last_change":"2024-05-03T17:52:35.732505+0000","last_active":"2024-05-03T17:53:19.870096+0000","last_peered":"2024-05-03T17:53:19.870096+0000","last_clean":"2024-05-03T17:53:19.870096+0000","last_became_active":"2024-05-03T17:52:35.732180+0000","last_became_peered":"2024-05-03T17:52:35.732180+0000","last_unstale":"2024-05-03T17:53:19.870096+0000","last_undegraded":"2024-05-03T17:53:19.870096+0000","last_fullsized":"2024-05-03T17:53:19.870096+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.371260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886349+0000","last_change":"2024-05-03T17:52:35.732217+0000","last_active":"2024-05-03T17:53:19.886349+0000","last_peered":"2024-05-03T17:53:19.886349+0000","last_clean":"2024-05-03T17:53:19.886349+0000","last_became_active":"2024-05-03T17:52:35.731868+0000","last_became_peered":"2024-05-03T17:52:35.731868+0000","last_unstale":"2024-05-03T17:53:19.886349+0000","last_undegraded":"2024-05-03T17:53:19.886349+0000","last_fullsized":"2024-05-03T17:53:19.886349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:04:17.521528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076813+0000","last_change":"2024-05-03T17:52:35.723641+0000","last_active":"2024-05-03T17:53:20.076813+0000","last_peered":"2024-05-03T17:53:20.076813+0000","last_clean":"2024-05-03T17:53:20.076813+0000","last_became_active":"2024-05-03T17:52:35.723323+0000","last_became_peered":"2024-05-03T17:52:35.723323+0000","last_unstale":"2024-05-03T17:53:20.076813+0000","last_undegraded":"2024-05-03T17:53:20.076813+0000","last_fullsized":"2024-05-03T17:53:20.076813+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:45:34.032407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872477+0000","last_change":"2024-05-03T17:52:35.731008+0000","last_active":"2024-05-03T17:53:19.872477+0000","last_peered":"2024-05-03T17:53:19.872477+0000","last_clean":"2024-05-03T17:53:19.872477+0000","last_became_active":"2024-05-03T17:52:35.730627+0000","last_became_peered":"2024-05-03T17:52:35.730627+0000","last_unstale":"2024-05-03T17:53:19.872477+0000","last_undegraded":"2024-05-03T17:53:19.872477+0000","last_fullsized":"2024-05-03T17:53:19.872477+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:30:32.884518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076519+0000","last_change":"2024-05-03T17:52:35.726938+0000","last_active":"2024-05-03T17:53:20.076519+0000","last_peered":"2024-05-03T17:53:20.076519+0000","last_clean":"2024-05-03T17:53:20.076519+0000","last_became_active":"2024-05-03T17:52:35.726611+0000","last_became_peered":"2024-05-03T17:52:35.726611+0000","last_unstale":"2024-05-03T17:53:20.076519+0000","last_undegraded":"2024-05-03T17:53:20.076519+0000","last_fullsized":"2024-05-03T17:53:20.076519+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:11.587035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867632+0000","last_change":"2024-05-03T17:52:35.722562+0000","last_active":"2024-05-03T17:53:19.867632+0000","last_peered":"2024-05-03T17:53:19.867632+0000","last_clean":"2024-05-03T17:53:19.867632+0000","last_became_active":"2024-05-03T17:52:35.722229+0000","last_became_peered":"2024-05-03T17:52:35.722229+0000","last_unstale":"2024-05-03T17:53:19.867632+0000","last_undegraded":"2024-05-03T17:53:19.867632+0000","last_fullsized":"2024-05-03T17:53:19.867632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:57:06.730533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867188+0000","last_change":"2024-05-03T17:52:35.722707+0000","last_active":"2024-05-03T17:53:19.867188+0000","last_peered":"2024-05-03T17:53:19.867188+0000","last_clean":"2024-05-03T17:53:19.867188+0000","last_became_active":"2024-05-03T17:52:35.721847+0000","last_became_peered":"2024-05-03T17:52:35.721847+0000","last_unstale":"2024-05-03T17:53:19.867188+0000","last_undegraded":"2024-05-03T17:53:19.867188+0000","last_fullsized":"2024-05-03T17:53:19.867188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:56:21.620074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870547+0000","last_change":"2024-05-03T17:52:35.731810+0000","last_active":"2024-05-03T17:53:19.870547+0000","last_peered":"2024-05-03T17:53:19.870547+0000","last_clean":"2024-05-03T17:53:19.870547+0000","last_became_active":"2024-05-03T17:52:35.728986+0000","last_became_peered":"2024-05-03T17:52:35.728986+0000","last_unstale":"2024-05-03T17:53:19.870547+0000","last_undegraded":"2024-05-03T17:53:19.870547+0000","last_fullsized":"2024-05-03T17:53:19.870547+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:18.275792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280811,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57557,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":207,"num_read_kb":163,"num_write":160,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":172671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743450,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":417972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1063009,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070501,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":418208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266729,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":418128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462958,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":1010449,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626483,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":417597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":470555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":418868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":472296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":872448,"data_stored":417908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986238,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":860160,"data_stored":418425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822713,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":1009742,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":53165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T17:53:27.385 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph status --format=json 2024-05-03T17:53:27.622 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:28.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:27 smithi122 ceph-mon[29744]: pgmap v313: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:28.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:27 smithi122 ceph-mon[29744]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:28.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:27 smithi149 ceph-mon[33409]: pgmap v313: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:28.287 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:27 smithi149 ceph-mon[33409]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:28.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:27 smithi192 ceph-mon[33753]: pgmap v313: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:28.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:27 smithi192 ceph-mon[33753]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:28.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:28 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:53:29.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.239+0000 7f03bf5fe700 1 -- 172.21.15.122:0/3993727880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b8102450 msgr2=0x7f03b8110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.239+0000 7f03bf5fe700 1 --2- 172.21.15.122:0/3993727880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b8102450 0x7f03b8110340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f03b000b0a0 tx=0x7f03b000b3b0 comp rx=0 tx=0).stop 2024-05-03T17:53:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.239+0000 7f03bf5fe700 1 -- 172.21.15.122:0/3993727880 shutdown_connections 2024-05-03T17:53:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.239+0000 7f03bf5fe700 1 --2- 172.21.15.122:0/3993727880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b8102450 0x7f03b8110340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.239+0000 7f03bf5fe700 1 --2- 172.21.15.122:0/3993727880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8109520 0x7f03b8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.239+0000 7f03bf5fe700 1 --2- 172.21.15.122:0/3993727880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8101a90 0x7f03b8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.239+0000 7f03bf5fe700 1 -- 172.21.15.122:0/3993727880 >> 172.21.15.122:0/3993727880 conn(0x7f03b8078a10 msgr2=0x7f03b80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.239+0000 7f03bf5fe700 1 -- 172.21.15.122:0/3993727880 shutdown_connections 2024-05-03T17:53:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.239+0000 7f03bf5fe700 1 -- 172.21.15.122:0/3993727880 wait complete. 2024-05-03T17:53:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.239+0000 7f03bf5fe700 1 Processor -- start 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bf5fe700 1 -- start start 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bf5fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8101a90 0x7f03b8102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bf5fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b8102450 0x7f03b81031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bf5fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8109520 0x7f03b81036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bf5fe700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03b8113650 con 0x7f03b8102450 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bf5fe700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03b81134d0 con 0x7f03b8101a90 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bf5fe700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f03b81137d0 con 0x7f03b8109520 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bcb99700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b8102450 0x7f03b81031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bcb99700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b8102450 0x7f03b81031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50992/0 (socket says 172.21.15.122:50992) 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bcb99700 1 -- 172.21.15.122:0/986603662 learned_addr learned my addr 172.21.15.122:0/986603662 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bd39a700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8101a90 0x7f03b8102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bdb9b700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8109520 0x7f03b81036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bd39a700 1 -- 172.21.15.122:0/986603662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8109520 msgr2=0x7f03b81036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bd39a700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8109520 0x7f03b81036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bd39a700 1 -- 172.21.15.122:0/986603662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b8102450 msgr2=0x7f03b81031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bd39a700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b8102450 0x7f03b81031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bd39a700 1 -- 172.21.15.122:0/986603662 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b81a8020 con 0x7f03b8101a90 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bcb99700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b8102450 0x7f03b81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.240+0000 7f03bdb9b700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8109520 0x7f03b81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.241+0000 7f03bd39a700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8101a90 0x7f03b8102c70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03b400e9f0 tx=0x7f03b400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.241+0000 7f03ae7fc700 1 -- 172.21.15.122:0/986603662 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4004220 con 0x7f03b8101a90 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.241+0000 7f03bf5fe700 1 -- 172.21.15.122:0/986603662 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b81a8310 con 0x7f03b8101a90 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.241+0000 7f03ae7fc700 1 -- 172.21.15.122:0/986603662 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b40043c0 con 0x7f03b8101a90 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.241+0000 7f03ae7fc700 1 -- 172.21.15.122:0/986603662 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4005020 con 0x7f03b8101a90 2024-05-03T17:53:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.241+0000 7f03bf5fe700 1 -- 172.21.15.122:0/986603662 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b81a8800 con 0x7f03b8101a90 2024-05-03T17:53:29.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.242+0000 7f03bf5fe700 1 -- 172.21.15.122:0/986603662 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b810ba00 con 0x7f03b8101a90 2024-05-03T17:53:29.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.244+0000 7f03ae7fc700 1 -- 172.21.15.122:0/986603662 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f03b4004750 con 0x7f03b8101a90 2024-05-03T17:53:29.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.244+0000 7f03ae7fc700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f03a40b2910 0x7f03a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:29.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.245+0000 7f03bcb99700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f03a40b2910 0x7f03a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:29.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.245+0000 7f03ae7fc700 1 -- 172.21.15.122:0/986603662 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f03b40e2170 con 0x7f03b8101a90 2024-05-03T17:53:29.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.245+0000 7f03bcb99700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f03a40b2910 0x7f03a40b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f03b8108c10 tx=0x7f03a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:29.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.247+0000 7f03ae7fc700 1 -- 172.21.15.122:0/986603662 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f03b40a9e30 con 0x7f03b8101a90 2024-05-03T17:53:29.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:28 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:53:29.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:28 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:53:29.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.568+0000 7f03bf5fe700 1 -- 172.21.15.122:0/986603662 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f03b8104150 con 0x7f03b8101a90 2024-05-03T17:53:29.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.568+0000 7f03ae7fc700 1 -- 172.21.15.122:0/986603662 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2461 (secure 0 0 0) 0x7f03b40a9730 con 0x7f03b8101a90 2024-05-03T17:53:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:53:29.563 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":433,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1714758688,"num_in_osds":12,"osd_in_since":1714758667,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":64,"data_bytes":648159,"bytes_used":385363968,"bytes_avail":1151555063808,"bytes_total":1151940427776,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"read_bytes_sec":3594,"write_bytes_sec":528,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":24,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"h","status":"up:standby-replay","gid":24410},{"filesystem_id":1,"rank":1,"name":"l","status":"up:standby-replay","gid":24421},{"filesystem_id":1,"rank":0,"name":"b","status":"up:standby-replay","gid":24443},{"filesystem_id":1,"rank":2,"name":"f","status":"up:active","gid":24457},{"filesystem_id":1,"rank":1,"name":"c","status":"up:active","gid":24463},{"filesystem_id":1,"rank":0,"name":"i","status":"up:active","gid":24481}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.122:8443/"}},"servicemap":{"epoch":25,"modified":"2024-05-03T17:53:18.548000+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-03T17:53:29.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.570+0000 7f03bf5fe700 1 -- 172.21.15.122:0/986603662 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f03a40b2910 msgr2=0x7f03a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:29.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.570+0000 7f03bf5fe700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f03a40b2910 0x7f03a40b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f03b8108c10 tx=0x7f03a8009260 comp rx=0 tx=0).stop 2024-05-03T17:53:29.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.570+0000 7f03bf5fe700 1 -- 172.21.15.122:0/986603662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8101a90 msgr2=0x7f03b8102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:29.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.570+0000 7f03bf5fe700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8101a90 0x7f03b8102c70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03b400e9f0 tx=0x7f03b400ed00 comp rx=0 tx=0).stop 2024-05-03T17:53:29.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.571+0000 7f03bf5fe700 1 -- 172.21.15.122:0/986603662 shutdown_connections 2024-05-03T17:53:29.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.571+0000 7f03bf5fe700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f03a40b2910 0x7f03a40b4dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:29.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.571+0000 7f03bf5fe700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8109520 0x7f03b81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:29.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.571+0000 7f03bf5fe700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8101a90 0x7f03b8102c70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:29.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.571+0000 7f03bf5fe700 1 --2- 172.21.15.122:0/986603662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b8102450 0x7f03b81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:29.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.571+0000 7f03bf5fe700 1 -- 172.21.15.122:0/986603662 >> 172.21.15.122:0/986603662 conn(0x7f03b8078a10 msgr2=0x7f03b8100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:29.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.571+0000 7f03bf5fe700 1 -- 172.21.15.122:0/986603662 shutdown_connections 2024-05-03T17:53:29.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:29.571+0000 7f03bf5fe700 1 -- 172.21.15.122:0/986603662 wait complete. 2024-05-03T17:53:29.921 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:29 smithi192 ceph-mon[33753]: pgmap v314: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 528 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:29.921 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:29 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/986603662' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-03T17:53:30.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:29 smithi149 ceph-mon[33409]: pgmap v314: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 528 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:30.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:29 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/986603662' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-03T17:53:30.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:29 smithi122 ceph-mon[29744]: pgmap v314: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 528 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:30.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:29 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/986603662' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-03T17:53:30.771 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-03T17:53:30.771 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T17:53:31.031 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:31.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:31 smithi122 ceph-mon[29744]: pgmap v315: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 479 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:31.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:31 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:31.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:31 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:31.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:31 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:31.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:31 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:31.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:31 smithi192 ceph-mon[33753]: pgmap v315: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 479 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:31.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:31 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:31.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:31 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:31.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:31 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:31.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:31 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:32.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:31 smithi149 ceph-mon[33409]: pgmap v315: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 479 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:32.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:31 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:32.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:31 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:32.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:31 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:32.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:31 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.702+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1808582576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f68103ef0 msgr2=0x7f5f681103a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.702+0000 7f5f6e0e2700 1 --2- 172.21.15.122:0/1808582576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f68103ef0 0x7f5f681103a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000b0d0 tx=0x7f5f6000b3e0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.702+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1808582576 shutdown_connections 2024-05-03T17:53:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.702+0000 7f5f6e0e2700 1 --2- 172.21.15.122:0/1808582576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f68103ef0 0x7f5f681103a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.702+0000 7f5f6e0e2700 1 --2- 172.21.15.122:0/1808582576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f681095d0 0x7f5f681099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.702+0000 7f5f6e0e2700 1 --2- 172.21.15.122:0/1808582576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f68103530 0x7f5f681039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.702+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1808582576 >> 172.21.15.122:0/1808582576 conn(0x7f5f6807a690 msgr2=0x7f5f6807aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.702+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1808582576 shutdown_connections 2024-05-03T17:53:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.702+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1808582576 wait complete. 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.702+0000 7f5f6e0e2700 1 Processor -- start 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f6e0e2700 1 -- start start 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f6e0e2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f68103530 0x7f5f6819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f6e0e2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f68103ef0 0x7f5f6819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f6e0e2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f681095d0 0x7f5f68199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f6e0e2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681136f0 con 0x7f5f68103530 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f6e0e2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68113570 con 0x7f5f681095d0 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f6e0e2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68113870 con 0x7f5f68103ef0 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f677fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f68103530 0x7f5f6819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f677fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f68103530 0x7f5f6819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51012/0 (socket says 172.21.15.122:51012) 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f677fe700 1 -- 172.21.15.122:0/1547741591 learned_addr learned my addr 172.21.15.122:0/1547741591 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f66ffd700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f68103ef0 0x7f5f6819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:32.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f67fff700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f681095d0 0x7f5f68199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:32.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f677fe700 1 -- 172.21.15.122:0/1547741591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f68103ef0 msgr2=0x7f5f6819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:32.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f677fe700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f68103ef0 0x7f5f6819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f677fe700 1 -- 172.21.15.122:0/1547741591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f681095d0 msgr2=0x7f5f68199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:32.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.703+0000 7f5f677fe700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f681095d0 0x7f5f68199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.704+0000 7f5f677fe700 1 -- 172.21.15.122:0/1547741591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6819a070 con 0x7f5f68103530 2024-05-03T17:53:32.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.704+0000 7f5f677fe700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f68103530 0x7f5f6819f750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800e9a0 tx=0x7f5f5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:32.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.704+0000 7f5f64ff9700 1 -- 172.21.15.122:0/1547741591 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800ca10 con 0x7f5f68103530 2024-05-03T17:53:32.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.704+0000 7f5f64ff9700 1 -- 172.21.15.122:0/1547741591 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5800cbb0 con 0x7f5f68103530 2024-05-03T17:53:32.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.704+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1547741591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6819a360 con 0x7f5f68103530 2024-05-03T17:53:32.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.704+0000 7f5f64ff9700 1 -- 172.21.15.122:0/1547741591 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58018680 con 0x7f5f68103530 2024-05-03T17:53:32.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.704+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1547741591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f681a6620 con 0x7f5f68103530 2024-05-03T17:53:32.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.705+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1547741591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6806a190 con 0x7f5f68103530 2024-05-03T17:53:32.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.707+0000 7f5f64ff9700 1 -- 172.21.15.122:0/1547741591 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f5f58010430 con 0x7f5f68103530 2024-05-03T17:53:32.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.707+0000 7f5f64ff9700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5f500b2910 0x7f5f500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:32.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.707+0000 7f5f66ffd700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5f500b2910 0x7f5f500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:32.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.707+0000 7f5f64ff9700 1 -- 172.21.15.122:0/1547741591 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f5f580e2750 con 0x7f5f68103530 2024-05-03T17:53:32.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.708+0000 7f5f66ffd700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5f500b2910 0x7f5f500b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5f6807ac90 tx=0x7f5f5c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:32.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.711+0000 7f5f64ff9700 1 -- 172.21.15.122:0/1547741591 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5f580aa410 con 0x7f5f68103530 2024-05-03T17:53:32.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.925+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1547741591 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f680638e0 con 0x7f5f500b2910 2024-05-03T17:53:32.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.932+0000 7f5f64ff9700 1 -- 172.21.15.122:0/1547741591 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258861 (secure 0 0 0) 0x7f5f680638e0 con 0x7f5f500b2910 2024-05-03T17:53:32.934 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:53:32.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1547741591 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5f500b2910 msgr2=0x7f5f500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:32.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5f500b2910 0x7f5f500b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5f6807ac90 tx=0x7f5f5c00b3c0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1547741591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f68103530 msgr2=0x7f5f6819f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:32.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f68103530 0x7f5f6819f750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800e9a0 tx=0x7f5f5800ecb0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1547741591 shutdown_connections 2024-05-03T17:53:32.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5f500b2910 0x7f5f500b4dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f681095d0 0x7f5f68199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f68103530 0x7f5f6819f750 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 --2- 172.21.15.122:0/1547741591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f68103ef0 0x7f5f6819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:32.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1547741591 >> 172.21.15.122:0/1547741591 conn(0x7f5f6807a690 msgr2=0x7f5f681021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:32.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1547741591 shutdown_connections 2024-05-03T17:53:32.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:32.935+0000 7f5f6e0e2700 1 -- 172.21.15.122:0/1547741591 wait complete. 2024-05-03T17:53:32.937 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-05-03T17:53:33.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:33.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:33.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:53:33.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:53:33.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:33.642 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:33.642 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:33.642 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:53:33.642 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:53:33.642 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:53:33.670 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-05-03T17:53:32.551314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649811,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":329,"num_read_kb":254,"num_write":286,"num_write_kb":2257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376336,"kb_used_data":10492,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565488,"statfs":{"total":1151940427776,"available":1151555059712,"internally_reserved":0,"allocated":10743808,"data_stored":6954809,"data_compressed":107130,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5216,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":46,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003030"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870114+0000","last_change":"2024-05-03T17:52:35.725057+0000","last_active":"2024-05-03T17:53:19.870114+0000","last_peered":"2024-05-03T17:53:19.870114+0000","last_clean":"2024-05-03T17:53:19.870114+0000","last_became_active":"2024-05-03T17:52:35.724632+0000","last_became_peered":"2024-05-03T17:52:35.724632+0000","last_unstale":"2024-05-03T17:53:19.870114+0000","last_undegraded":"2024-05-03T17:53:19.870114+0000","last_fullsized":"2024-05-03T17:53:19.870114+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:46.065357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870650+0000","last_change":"2024-05-03T17:52:35.728500+0000","last_active":"2024-05-03T17:53:19.870650+0000","last_peered":"2024-05-03T17:53:19.870650+0000","last_clean":"2024-05-03T17:53:19.870650+0000","last_became_active":"2024-05-03T17:52:35.728210+0000","last_became_peered":"2024-05-03T17:52:35.728210+0000","last_unstale":"2024-05-03T17:53:19.870650+0000","last_undegraded":"2024-05-03T17:53:19.870650+0000","last_fullsized":"2024-05-03T17:53:19.870650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:33:23.410617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870644+0000","last_change":"2024-05-03T17:52:35.726300+0000","last_active":"2024-05-03T17:53:19.870644+0000","last_peered":"2024-05-03T17:53:19.870644+0000","last_clean":"2024-05-03T17:53:19.870644+0000","last_became_active":"2024-05-03T17:52:35.725994+0000","last_became_peered":"2024-05-03T17:52:35.725994+0000","last_unstale":"2024-05-03T17:53:19.870644+0000","last_undegraded":"2024-05-03T17:53:19.870644+0000","last_fullsized":"2024-05-03T17:53:19.870644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:31:19.723453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077278+0000","last_change":"2024-05-03T17:52:35.728105+0000","last_active":"2024-05-03T17:53:20.077278+0000","last_peered":"2024-05-03T17:53:20.077278+0000","last_clean":"2024-05-03T17:53:20.077278+0000","last_became_active":"2024-05-03T17:52:35.727762+0000","last_became_peered":"2024-05-03T17:52:35.727762+0000","last_unstale":"2024-05-03T17:53:20.077278+0000","last_undegraded":"2024-05-03T17:53:20.077278+0000","last_fullsized":"2024-05-03T17:53:20.077278+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.640057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870597+0000","last_change":"2024-05-03T17:52:35.734516+0000","last_active":"2024-05-03T17:53:19.870597+0000","last_peered":"2024-05-03T17:53:19.870597+0000","last_clean":"2024-05-03T17:53:19.870597+0000","last_became_active":"2024-05-03T17:52:35.734125+0000","last_became_peered":"2024-05-03T17:52:35.734125+0000","last_unstale":"2024-05-03T17:53:19.870597+0000","last_undegraded":"2024-05-03T17:53:19.870597+0000","last_fullsized":"2024-05-03T17:53:19.870597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:39:25.103137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871539+0000","last_change":"2024-05-03T17:52:35.723427+0000","last_active":"2024-05-03T17:53:19.871539+0000","last_peered":"2024-05-03T17:53:19.871539+0000","last_clean":"2024-05-03T17:53:19.871539+0000","last_became_active":"2024-05-03T17:52:35.723113+0000","last_became_peered":"2024-05-03T17:52:35.723113+0000","last_unstale":"2024-05-03T17:53:19.871539+0000","last_undegraded":"2024-05-03T17:53:19.871539+0000","last_fullsized":"2024-05-03T17:53:19.871539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:37:14.606347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077441+0000","last_change":"2024-05-03T17:52:35.730858+0000","last_active":"2024-05-03T17:53:20.077441+0000","last_peered":"2024-05-03T17:53:20.077441+0000","last_clean":"2024-05-03T17:53:20.077441+0000","last_became_active":"2024-05-03T17:52:35.730453+0000","last_became_peered":"2024-05-03T17:52:35.730453+0000","last_unstale":"2024-05-03T17:53:20.077441+0000","last_undegraded":"2024-05-03T17:53:20.077441+0000","last_fullsized":"2024-05-03T17:53:20.077441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:21:17.803027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076565+0000","last_change":"2024-05-03T17:52:35.730547+0000","last_active":"2024-05-03T17:53:20.076565+0000","last_peered":"2024-05-03T17:53:20.076565+0000","last_clean":"2024-05-03T17:53:20.076565+0000","last_became_active":"2024-05-03T17:52:35.729582+0000","last_became_peered":"2024-05-03T17:52:35.729582+0000","last_unstale":"2024-05-03T17:53:20.076565+0000","last_undegraded":"2024-05-03T17:53:20.076565+0000","last_fullsized":"2024-05-03T17:53:20.076565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:24:55.110537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.868010+0000","last_change":"2024-05-03T17:52:35.726305+0000","last_active":"2024-05-03T17:53:19.868010+0000","last_peered":"2024-05-03T17:53:19.868010+0000","last_clean":"2024-05-03T17:53:19.868010+0000","last_became_active":"2024-05-03T17:52:35.725858+0000","last_became_peered":"2024-05-03T17:52:35.725858+0000","last_unstale":"2024-05-03T17:53:19.868010+0000","last_undegraded":"2024-05-03T17:53:19.868010+0000","last_fullsized":"2024-05-03T17:53:19.868010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:59.347307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871846+0000","last_change":"2024-05-03T17:52:35.726865+0000","last_active":"2024-05-03T17:53:19.871846+0000","last_peered":"2024-05-03T17:53:19.871846+0000","last_clean":"2024-05-03T17:53:19.871846+0000","last_became_active":"2024-05-03T17:52:35.726483+0000","last_became_peered":"2024-05-03T17:52:35.726483+0000","last_unstale":"2024-05-03T17:53:19.871846+0000","last_undegraded":"2024-05-03T17:53:19.871846+0000","last_fullsized":"2024-05-03T17:53:19.871846+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:17.851133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077159+0000","last_change":"2024-05-03T17:52:35.730551+0000","last_active":"2024-05-03T17:53:20.077159+0000","last_peered":"2024-05-03T17:53:20.077159+0000","last_clean":"2024-05-03T17:53:20.077159+0000","last_became_active":"2024-05-03T17:52:35.730187+0000","last_became_peered":"2024-05-03T17:52:35.730187+0000","last_unstale":"2024-05-03T17:53:20.077159+0000","last_undegraded":"2024-05-03T17:53:20.077159+0000","last_fullsized":"2024-05-03T17:53:20.077159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:18:56.211061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077071+0000","last_change":"2024-05-03T17:52:35.730040+0000","last_active":"2024-05-03T17:53:20.077071+0000","last_peered":"2024-05-03T17:53:20.077071+0000","last_clean":"2024-05-03T17:53:20.077071+0000","last_became_active":"2024-05-03T17:52:35.729549+0000","last_became_peered":"2024-05-03T17:52:35.729549+0000","last_unstale":"2024-05-03T17:53:20.077071+0000","last_undegraded":"2024-05-03T17:53:20.077071+0000","last_fullsized":"2024-05-03T17:53:20.077071+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:50:05.052247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867289+0000","last_change":"2024-05-03T17:52:35.726709+0000","last_active":"2024-05-03T17:53:19.867289+0000","last_peered":"2024-05-03T17:53:19.867289+0000","last_clean":"2024-05-03T17:53:19.867289+0000","last_became_active":"2024-05-03T17:52:35.726166+0000","last_became_peered":"2024-05-03T17:52:35.726166+0000","last_unstale":"2024-05-03T17:53:19.867289+0000","last_undegraded":"2024-05-03T17:53:19.867289+0000","last_fullsized":"2024-05-03T17:53:19.867289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.164622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870439+0000","last_change":"2024-05-03T17:52:35.727583+0000","last_active":"2024-05-03T17:53:19.870439+0000","last_peered":"2024-05-03T17:53:19.870439+0000","last_clean":"2024-05-03T17:53:19.870439+0000","last_became_active":"2024-05-03T17:52:35.727131+0000","last_became_peered":"2024-05-03T17:52:35.727131+0000","last_unstale":"2024-05-03T17:53:19.870439+0000","last_undegraded":"2024-05-03T17:53:19.870439+0000","last_fullsized":"2024-05-03T17:53:19.870439+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:28:21.255954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.887563+0000","last_change":"2024-05-03T17:53:19.887563+0000","last_active":"2024-05-03T17:53:19.887563+0000","last_peered":"2024-05-03T17:53:19.887563+0000","last_clean":"2024-05-03T17:53:19.887563+0000","last_became_active":"2024-05-03T17:53:19.887068+0000","last_became_peered":"2024-05-03T17:53:19.887068+0000","last_unstale":"2024-05-03T17:53:19.887563+0000","last_undegraded":"2024-05-03T17:53:19.887563+0000","last_fullsized":"2024-05-03T17:53:19.887563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:49:49.354138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867360+0000","last_change":"2024-05-03T17:52:35.717354+0000","last_active":"2024-05-03T17:53:19.867360+0000","last_peered":"2024-05-03T17:53:19.867360+0000","last_clean":"2024-05-03T17:53:19.867360+0000","last_became_active":"2024-05-03T17:52:35.717025+0000","last_became_peered":"2024-05-03T17:52:35.717025+0000","last_unstale":"2024-05-03T17:53:19.867360+0000","last_undegraded":"2024-05-03T17:53:19.867360+0000","last_fullsized":"2024-05-03T17:53:19.867360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:11:07.778484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870387+0000","last_change":"2024-05-03T17:52:35.725626+0000","last_active":"2024-05-03T17:53:19.870387+0000","last_peered":"2024-05-03T17:53:19.870387+0000","last_clean":"2024-05-03T17:53:19.870387+0000","last_became_active":"2024-05-03T17:52:35.725239+0000","last_became_peered":"2024-05-03T17:52:35.725239+0000","last_unstale":"2024-05-03T17:53:19.870387+0000","last_undegraded":"2024-05-03T17:53:19.870387+0000","last_fullsized":"2024-05-03T17:53:19.870387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:44:18.359484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869622+0000","last_change":"2024-05-03T17:52:35.724250+0000","last_active":"2024-05-03T17:53:19.869622+0000","last_peered":"2024-05-03T17:53:19.869622+0000","last_clean":"2024-05-03T17:53:19.869622+0000","last_became_active":"2024-05-03T17:52:35.723998+0000","last_became_peered":"2024-05-03T17:52:35.723998+0000","last_unstale":"2024-05-03T17:53:19.869622+0000","last_undegraded":"2024-05-03T17:53:19.869622+0000","last_fullsized":"2024-05-03T17:53:19.869622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:03:32.939559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886262+0000","last_change":"2024-05-03T17:52:35.726237+0000","last_active":"2024-05-03T17:53:19.886262+0000","last_peered":"2024-05-03T17:53:19.886262+0000","last_clean":"2024-05-03T17:53:19.886262+0000","last_became_active":"2024-05-03T17:52:35.725847+0000","last_became_peered":"2024-05-03T17:52:35.725847+0000","last_unstale":"2024-05-03T17:53:19.886262+0000","last_undegraded":"2024-05-03T17:53:19.886262+0000","last_fullsized":"2024-05-03T17:53:19.886262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:06.472532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869723+0000","last_change":"2024-05-03T17:52:32.687198+0000","last_active":"2024-05-03T17:53:19.869723+0000","last_peered":"2024-05-03T17:53:19.869723+0000","last_clean":"2024-05-03T17:53:19.869723+0000","last_became_active":"2024-05-03T17:52:32.686848+0000","last_became_peered":"2024-05-03T17:52:32.686848+0000","last_unstale":"2024-05-03T17:53:19.869723+0000","last_undegraded":"2024-05-03T17:53:19.869723+0000","last_fullsized":"2024-05-03T17:53:19.869723+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:08:24.244371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076745+0000","last_change":"2024-05-03T17:52:35.730595+0000","last_active":"2024-05-03T17:53:20.076745+0000","last_peered":"2024-05-03T17:53:20.076745+0000","last_clean":"2024-05-03T17:53:20.076745+0000","last_became_active":"2024-05-03T17:52:35.729999+0000","last_became_peered":"2024-05-03T17:52:35.729999+0000","last_unstale":"2024-05-03T17:53:20.076745+0000","last_undegraded":"2024-05-03T17:53:20.076745+0000","last_fullsized":"2024-05-03T17:53:20.076745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:14.485756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076923+0000","last_change":"2024-05-03T17:52:32.935239+0000","last_active":"2024-05-03T17:53:20.076923+0000","last_peered":"2024-05-03T17:53:20.076923+0000","last_clean":"2024-05-03T17:53:20.076923+0000","last_became_active":"2024-05-03T17:52:32.934915+0000","last_became_peered":"2024-05-03T17:52:32.934915+0000","last_unstale":"2024-05-03T17:53:20.076923+0000","last_undegraded":"2024-05-03T17:53:20.076923+0000","last_fullsized":"2024-05-03T17:53:20.076923+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:20.785071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870934+0000","last_change":"2024-05-03T17:52:35.724940+0000","last_active":"2024-05-03T17:53:19.870934+0000","last_peered":"2024-05-03T17:53:19.870934+0000","last_clean":"2024-05-03T17:53:19.870934+0000","last_became_active":"2024-05-03T17:52:35.724603+0000","last_became_peered":"2024-05-03T17:52:35.724603+0000","last_unstale":"2024-05-03T17:53:19.870934+0000","last_undegraded":"2024-05-03T17:53:19.870934+0000","last_fullsized":"2024-05-03T17:53:19.870934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:43.640499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871451+0000","last_change":"2024-05-03T17:52:32.688202+0000","last_active":"2024-05-03T17:53:19.871451+0000","last_peered":"2024-05-03T17:53:19.871451+0000","last_clean":"2024-05-03T17:53:19.871451+0000","last_became_active":"2024-05-03T17:52:32.687893+0000","last_became_peered":"2024-05-03T17:52:32.687893+0000","last_unstale":"2024-05-03T17:53:19.871451+0000","last_undegraded":"2024-05-03T17:53:19.871451+0000","last_fullsized":"2024-05-03T17:53:19.871451+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:43.342832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867555+0000","last_change":"2024-05-03T17:52:35.720980+0000","last_active":"2024-05-03T17:53:19.867555+0000","last_peered":"2024-05-03T17:53:19.867555+0000","last_clean":"2024-05-03T17:53:19.867555+0000","last_became_active":"2024-05-03T17:52:35.720582+0000","last_became_peered":"2024-05-03T17:52:35.720582+0000","last_unstale":"2024-05-03T17:53:19.867555+0000","last_undegraded":"2024-05-03T17:53:19.867555+0000","last_fullsized":"2024-05-03T17:53:19.867555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:01:22.938186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.877543+0000","last_change":"2024-05-03T17:53:19.877543+0000","last_active":"2024-05-03T17:53:19.877543+0000","last_peered":"2024-05-03T17:53:19.877543+0000","last_clean":"2024-05-03T17:53:19.877543+0000","last_became_active":"2024-05-03T17:53:19.873988+0000","last_became_peered":"2024-05-03T17:53:19.873988+0000","last_unstale":"2024-05-03T17:53:19.877543+0000","last_undegraded":"2024-05-03T17:53:19.877543+0000","last_fullsized":"2024-05-03T17:53:19.877543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:49:49.105623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.079283+0000","last_change":"2024-05-03T17:53:20.078906+0000","last_active":"2024-05-03T17:53:20.079283+0000","last_peered":"2024-05-03T17:53:20.079283+0000","last_clean":"2024-05-03T17:53:20.079283+0000","last_became_active":"2024-05-03T17:53:20.078538+0000","last_became_peered":"2024-05-03T17:53:20.078538+0000","last_unstale":"2024-05-03T17:53:20.079283+0000","last_undegraded":"2024-05-03T17:53:20.079283+0000","last_fullsized":"2024-05-03T17:53:20.079283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:59:46.945417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":70,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:27.816133+0000","last_change":"2024-05-03T17:52:32.692735+0000","last_active":"2024-05-03T17:53:27.816133+0000","last_peered":"2024-05-03T17:53:27.816133+0000","last_clean":"2024-05-03T17:53:27.816133+0000","last_became_active":"2024-05-03T17:52:32.692394+0000","last_became_peered":"2024-05-03T17:52:32.692394+0000","last_unstale":"2024-05-03T17:53:27.816133+0000","last_undegraded":"2024-05-03T17:53:27.816133+0000","last_fullsized":"2024-05-03T17:53:27.816133+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:40:11.434751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870798+0000","last_change":"2024-05-03T17:52:35.727167+0000","last_active":"2024-05-03T17:53:19.870798+0000","last_peered":"2024-05-03T17:53:19.870798+0000","last_clean":"2024-05-03T17:53:19.870798+0000","last_became_active":"2024-05-03T17:52:35.726792+0000","last_became_peered":"2024-05-03T17:52:35.726792+0000","last_unstale":"2024-05-03T17:53:19.870798+0000","last_undegraded":"2024-05-03T17:53:19.870798+0000","last_fullsized":"2024-05-03T17:53:19.870798+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:54:01.823025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871452+0000","last_change":"2024-05-03T17:52:32.687573+0000","last_active":"2024-05-03T17:53:19.871452+0000","last_peered":"2024-05-03T17:53:19.871452+0000","last_clean":"2024-05-03T17:53:19.871452+0000","last_became_active":"2024-05-03T17:52:32.686158+0000","last_became_peered":"2024-05-03T17:52:32.686158+0000","last_unstale":"2024-05-03T17:53:19.871452+0000","last_undegraded":"2024-05-03T17:53:19.871452+0000","last_fullsized":"2024-05-03T17:53:19.871452+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:19.891479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873172+0000","last_change":"2024-05-03T17:52:35.728049+0000","last_active":"2024-05-03T17:53:19.873172+0000","last_peered":"2024-05-03T17:53:19.873172+0000","last_clean":"2024-05-03T17:53:19.873172+0000","last_became_active":"2024-05-03T17:52:35.727713+0000","last_became_peered":"2024-05-03T17:52:35.727713+0000","last_unstale":"2024-05-03T17:53:19.873172+0000","last_undegraded":"2024-05-03T17:53:19.873172+0000","last_fullsized":"2024-05-03T17:53:19.873172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:28:01.798457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867118+0000","last_change":"2024-05-03T17:52:32.685679+0000","last_active":"2024-05-03T17:53:19.867118+0000","last_peered":"2024-05-03T17:53:19.867118+0000","last_clean":"2024-05-03T17:53:19.867118+0000","last_became_active":"2024-05-03T17:52:32.685306+0000","last_became_peered":"2024-05-03T17:52:32.685306+0000","last_unstale":"2024-05-03T17:53:19.867118+0000","last_undegraded":"2024-05-03T17:53:19.867118+0000","last_fullsized":"2024-05-03T17:53:19.867118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:48:22.218310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076501+0000","last_change":"2024-05-03T17:52:35.717111+0000","last_active":"2024-05-03T17:53:20.076501+0000","last_peered":"2024-05-03T17:53:20.076501+0000","last_clean":"2024-05-03T17:53:20.076501+0000","last_became_active":"2024-05-03T17:52:35.716782+0000","last_became_peered":"2024-05-03T17:52:35.716782+0000","last_unstale":"2024-05-03T17:53:20.076501+0000","last_undegraded":"2024-05-03T17:53:20.076501+0000","last_fullsized":"2024-05-03T17:53:20.076501+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:31:57.766927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872477+0000","last_change":"2024-05-03T17:52:32.690846+0000","last_active":"2024-05-03T17:53:19.872477+0000","last_peered":"2024-05-03T17:53:19.872477+0000","last_clean":"2024-05-03T17:53:19.872477+0000","last_became_active":"2024-05-03T17:52:32.690506+0000","last_became_peered":"2024-05-03T17:52:32.690506+0000","last_unstale":"2024-05-03T17:53:19.872477+0000","last_undegraded":"2024-05-03T17:53:19.872477+0000","last_fullsized":"2024-05-03T17:53:19.872477+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:19.810205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869511+0000","last_change":"2024-05-03T17:52:35.718229+0000","last_active":"2024-05-03T17:53:19.869511+0000","last_peered":"2024-05-03T17:53:19.869511+0000","last_clean":"2024-05-03T17:53:19.869511+0000","last_became_active":"2024-05-03T17:52:35.717907+0000","last_became_peered":"2024-05-03T17:52:35.717907+0000","last_unstale":"2024-05-03T17:53:19.869511+0000","last_undegraded":"2024-05-03T17:53:19.869511+0000","last_fullsized":"2024-05-03T17:53:19.869511+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:45.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":68,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:30.819091+0000","last_change":"2024-05-03T17:52:32.691174+0000","last_active":"2024-05-03T17:53:30.819091+0000","last_peered":"2024-05-03T17:53:30.819091+0000","last_clean":"2024-05-03T17:53:30.819091+0000","last_became_active":"2024-05-03T17:52:32.689907+0000","last_became_peered":"2024-05-03T17:52:32.689907+0000","last_unstale":"2024-05-03T17:53:30.819091+0000","last_undegraded":"2024-05-03T17:53:30.819091+0000","last_fullsized":"2024-05-03T17:53:30.819091+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:46.630436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867946+0000","last_change":"2024-05-03T17:52:35.720290+0000","last_active":"2024-05-03T17:53:19.867946+0000","last_peered":"2024-05-03T17:53:19.867946+0000","last_clean":"2024-05-03T17:53:19.867946+0000","last_became_active":"2024-05-03T17:52:35.719962+0000","last_became_peered":"2024-05-03T17:52:35.719962+0000","last_unstale":"2024-05-03T17:53:19.867946+0000","last_undegraded":"2024-05-03T17:53:19.867946+0000","last_fullsized":"2024-05-03T17:53:19.867946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:31:41.301592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077710+0000","last_change":"2024-05-03T17:52:32.689937+0000","last_active":"2024-05-03T17:53:20.077710+0000","last_peered":"2024-05-03T17:53:20.077710+0000","last_clean":"2024-05-03T17:53:20.077710+0000","last_became_active":"2024-05-03T17:52:32.689624+0000","last_became_peered":"2024-05-03T17:52:32.689624+0000","last_unstale":"2024-05-03T17:53:20.077710+0000","last_undegraded":"2024-05-03T17:53:20.077710+0000","last_fullsized":"2024-05-03T17:53:20.077710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:24.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870881+0000","last_change":"2024-05-03T17:52:35.726824+0000","last_active":"2024-05-03T17:53:19.870881+0000","last_peered":"2024-05-03T17:53:19.870881+0000","last_clean":"2024-05-03T17:53:19.870881+0000","last_became_active":"2024-05-03T17:52:35.726494+0000","last_became_peered":"2024-05-03T17:52:35.726494+0000","last_unstale":"2024-05-03T17:53:19.870881+0000","last_undegraded":"2024-05-03T17:53:19.870881+0000","last_fullsized":"2024-05-03T17:53:19.870881+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:41:21.284852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873051+0000","last_change":"2024-05-03T17:52:32.935471+0000","last_active":"2024-05-03T17:53:19.873051+0000","last_peered":"2024-05-03T17:53:19.873051+0000","last_clean":"2024-05-03T17:53:19.873051+0000","last_became_active":"2024-05-03T17:52:32.935012+0000","last_became_peered":"2024-05-03T17:52:32.935012+0000","last_unstale":"2024-05-03T17:53:19.873051+0000","last_undegraded":"2024-05-03T17:53:19.873051+0000","last_fullsized":"2024-05-03T17:53:19.873051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:35:38.654577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867634+0000","last_change":"2024-05-03T17:52:35.727124+0000","last_active":"2024-05-03T17:53:19.867634+0000","last_peered":"2024-05-03T17:53:19.867634+0000","last_clean":"2024-05-03T17:53:19.867634+0000","last_became_active":"2024-05-03T17:52:35.726804+0000","last_became_peered":"2024-05-03T17:52:35.726804+0000","last_unstale":"2024-05-03T17:53:19.867634+0000","last_undegraded":"2024-05-03T17:53:19.867634+0000","last_fullsized":"2024-05-03T17:53:19.867634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:32.000803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'29","reported_seq":142,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:31.821001+0000","last_change":"2024-05-03T17:52:32.685014+0000","last_active":"2024-05-03T17:53:31.821001+0000","last_peered":"2024-05-03T17:53:31.821001+0000","last_clean":"2024-05-03T17:53:31.821001+0000","last_became_active":"2024-05-03T17:52:32.684660+0000","last_became_peered":"2024-05-03T17:52:32.684660+0000","last_unstale":"2024-05-03T17:53:31.821001+0000","last_undegraded":"2024-05-03T17:53:31.821001+0000","last_fullsized":"2024-05-03T17:53:31.821001+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:25:56.890283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54817,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":87,"num_read_kb":96,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871281+0000","last_change":"2024-05-03T17:52:35.733675+0000","last_active":"2024-05-03T17:53:19.871281+0000","last_peered":"2024-05-03T17:53:19.871281+0000","last_clean":"2024-05-03T17:53:19.871281+0000","last_became_active":"2024-05-03T17:52:35.733234+0000","last_became_peered":"2024-05-03T17:52:35.733234+0000","last_unstale":"2024-05-03T17:53:19.871281+0000","last_undegraded":"2024-05-03T17:53:19.871281+0000","last_fullsized":"2024-05-03T17:53:19.871281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:57.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886182+0000","last_change":"2024-05-03T17:52:32.690462+0000","last_active":"2024-05-03T17:53:19.886182+0000","last_peered":"2024-05-03T17:53:19.886182+0000","last_clean":"2024-05-03T17:53:19.886182+0000","last_became_active":"2024-05-03T17:52:32.690169+0000","last_became_peered":"2024-05-03T17:52:32.690169+0000","last_unstale":"2024-05-03T17:53:19.886182+0000","last_undegraded":"2024-05-03T17:53:19.886182+0000","last_fullsized":"2024-05-03T17:53:19.886182+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:59.324612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867010+0000","last_change":"2024-05-03T17:52:35.720730+0000","last_active":"2024-05-03T17:53:19.867010+0000","last_peered":"2024-05-03T17:53:19.867010+0000","last_clean":"2024-05-03T17:53:19.867010+0000","last_became_active":"2024-05-03T17:52:35.720392+0000","last_became_peered":"2024-05-03T17:52:35.720392+0000","last_unstale":"2024-05-03T17:53:19.867010+0000","last_undegraded":"2024-05-03T17:53:19.867010+0000","last_fullsized":"2024-05-03T17:53:19.867010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:58.528148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872742+0000","last_change":"2024-05-03T17:52:32.691439+0000","last_active":"2024-05-03T17:53:19.872742+0000","last_peered":"2024-05-03T17:53:19.872742+0000","last_clean":"2024-05-03T17:53:19.872742+0000","last_became_active":"2024-05-03T17:52:32.691080+0000","last_became_peered":"2024-05-03T17:52:32.691080+0000","last_unstale":"2024-05-03T17:53:19.872742+0000","last_undegraded":"2024-05-03T17:53:19.872742+0000","last_fullsized":"2024-05-03T17:53:19.872742+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:40:43.120194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077160+0000","last_change":"2024-05-03T17:52:35.729860+0000","last_active":"2024-05-03T17:53:20.077160+0000","last_peered":"2024-05-03T17:53:20.077160+0000","last_clean":"2024-05-03T17:53:20.077160+0000","last_became_active":"2024-05-03T17:52:35.729508+0000","last_became_peered":"2024-05-03T17:52:35.729508+0000","last_unstale":"2024-05-03T17:53:20.077160+0000","last_undegraded":"2024-05-03T17:53:20.077160+0000","last_fullsized":"2024-05-03T17:53:20.077160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:39:20.859215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":69,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:27.815914+0000","last_change":"2024-05-03T17:52:32.689346+0000","last_active":"2024-05-03T17:53:27.815914+0000","last_peered":"2024-05-03T17:53:27.815914+0000","last_clean":"2024-05-03T17:53:27.815914+0000","last_became_active":"2024-05-03T17:52:32.689009+0000","last_became_peered":"2024-05-03T17:52:32.689009+0000","last_unstale":"2024-05-03T17:53:27.815914+0000","last_undegraded":"2024-05-03T17:53:27.815914+0000","last_fullsized":"2024-05-03T17:53:27.815914+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:55:36.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077075+0000","last_change":"2024-05-03T17:52:35.729226+0000","last_active":"2024-05-03T17:53:20.077075+0000","last_peered":"2024-05-03T17:53:20.077075+0000","last_clean":"2024-05-03T17:53:20.077075+0000","last_became_active":"2024-05-03T17:52:35.728846+0000","last_became_peered":"2024-05-03T17:52:35.728846+0000","last_unstale":"2024-05-03T17:53:20.077075+0000","last_undegraded":"2024-05-03T17:53:20.077075+0000","last_fullsized":"2024-05-03T17:53:20.077075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:39:16.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870410+0000","last_change":"2024-05-03T17:52:32.936108+0000","last_active":"2024-05-03T17:53:19.870410+0000","last_peered":"2024-05-03T17:53:19.870410+0000","last_clean":"2024-05-03T17:53:19.870410+0000","last_became_active":"2024-05-03T17:52:32.935782+0000","last_became_peered":"2024-05-03T17:52:32.935782+0000","last_unstale":"2024-05-03T17:53:19.870410+0000","last_undegraded":"2024-05-03T17:53:19.870410+0000","last_fullsized":"2024-05-03T17:53:19.870410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:47.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872695+0000","last_change":"2024-05-03T17:52:35.727163+0000","last_active":"2024-05-03T17:53:19.872695+0000","last_peered":"2024-05-03T17:53:19.872695+0000","last_clean":"2024-05-03T17:53:19.872695+0000","last_became_active":"2024-05-03T17:52:35.726824+0000","last_became_peered":"2024-05-03T17:52:35.726824+0000","last_unstale":"2024-05-03T17:53:19.872695+0000","last_undegraded":"2024-05-03T17:53:19.872695+0000","last_fullsized":"2024-05-03T17:53:19.872695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:24:50.803860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871546+0000","last_change":"2024-05-03T17:52:32.687329+0000","last_active":"2024-05-03T17:53:19.871546+0000","last_peered":"2024-05-03T17:53:19.871546+0000","last_clean":"2024-05-03T17:53:19.871546+0000","last_became_active":"2024-05-03T17:52:32.687025+0000","last_became_peered":"2024-05-03T17:52:32.687025+0000","last_unstale":"2024-05-03T17:53:19.871546+0000","last_undegraded":"2024-05-03T17:53:19.871546+0000","last_fullsized":"2024-05-03T17:53:19.871546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:42:15.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869955+0000","last_change":"2024-05-03T17:52:35.728797+0000","last_active":"2024-05-03T17:53:19.869955+0000","last_peered":"2024-05-03T17:53:19.869955+0000","last_clean":"2024-05-03T17:53:19.869955+0000","last_became_active":"2024-05-03T17:52:35.728429+0000","last_became_peered":"2024-05-03T17:52:35.728429+0000","last_unstale":"2024-05-03T17:53:19.869955+0000","last_undegraded":"2024-05-03T17:53:19.869955+0000","last_fullsized":"2024-05-03T17:53:19.869955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:19.584593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872508+0000","last_change":"2024-05-03T17:53:19.872508+0000","last_active":"2024-05-03T17:53:19.872508+0000","last_peered":"2024-05-03T17:53:19.872508+0000","last_clean":"2024-05-03T17:53:19.872508+0000","last_became_active":"2024-05-03T17:53:19.872175+0000","last_became_peered":"2024-05-03T17:53:19.872175+0000","last_unstale":"2024-05-03T17:53:19.872508+0000","last_undegraded":"2024-05-03T17:53:19.872508+0000","last_fullsized":"2024-05-03T17:53:19.872508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:24:08.246884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871588+0000","last_change":"2024-05-03T17:52:35.719115+0000","last_active":"2024-05-03T17:53:19.871588+0000","last_peered":"2024-05-03T17:53:19.871588+0000","last_clean":"2024-05-03T17:53:19.871588+0000","last_became_active":"2024-05-03T17:52:35.718054+0000","last_became_peered":"2024-05-03T17:52:35.718054+0000","last_unstale":"2024-05-03T17:53:19.871588+0000","last_undegraded":"2024-05-03T17:53:19.871588+0000","last_fullsized":"2024-05-03T17:53:19.871588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.903766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077572+0000","last_change":"2024-05-03T17:52:32.688993+0000","last_active":"2024-05-03T17:53:20.077572+0000","last_peered":"2024-05-03T17:53:20.077572+0000","last_clean":"2024-05-03T17:53:20.077572+0000","last_became_active":"2024-05-03T17:52:32.688706+0000","last_became_peered":"2024-05-03T17:52:32.688706+0000","last_unstale":"2024-05-03T17:53:20.077572+0000","last_undegraded":"2024-05-03T17:53:20.077572+0000","last_fullsized":"2024-05-03T17:53:20.077572+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:40:14.023338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872957+0000","last_change":"2024-05-03T17:53:19.872957+0000","last_active":"2024-05-03T17:53:19.872957+0000","last_peered":"2024-05-03T17:53:19.872957+0000","last_clean":"2024-05-03T17:53:19.872957+0000","last_became_active":"2024-05-03T17:53:19.872690+0000","last_became_peered":"2024-05-03T17:53:19.872690+0000","last_unstale":"2024-05-03T17:53:19.872957+0000","last_undegraded":"2024-05-03T17:53:19.872957+0000","last_fullsized":"2024-05-03T17:53:19.872957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:53:43.801983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872998+0000","last_change":"2024-05-03T17:52:35.728958+0000","last_active":"2024-05-03T17:53:19.872998+0000","last_peered":"2024-05-03T17:53:19.872998+0000","last_clean":"2024-05-03T17:53:19.872998+0000","last_became_active":"2024-05-03T17:52:35.728619+0000","last_became_peered":"2024-05-03T17:52:35.728619+0000","last_unstale":"2024-05-03T17:53:19.872998+0000","last_undegraded":"2024-05-03T17:53:19.872998+0000","last_fullsized":"2024-05-03T17:53:19.872998+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:29:29.922324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:26.813970+0000","last_change":"2024-05-03T17:52:32.935367+0000","last_active":"2024-05-03T17:53:26.813970+0000","last_peered":"2024-05-03T17:53:26.813970+0000","last_clean":"2024-05-03T17:53:26.813970+0000","last_became_active":"2024-05-03T17:52:32.934938+0000","last_became_peered":"2024-05-03T17:52:32.934938+0000","last_unstale":"2024-05-03T17:53:26.813970+0000","last_undegraded":"2024-05-03T17:53:26.813970+0000","last_fullsized":"2024-05-03T17:53:26.813970+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:07:21.642883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076663+0000","last_change":"2024-05-03T17:52:35.731035+0000","last_active":"2024-05-03T17:53:20.076663+0000","last_peered":"2024-05-03T17:53:20.076663+0000","last_clean":"2024-05-03T17:53:20.076663+0000","last_became_active":"2024-05-03T17:52:35.730034+0000","last_became_peered":"2024-05-03T17:52:35.730034+0000","last_unstale":"2024-05-03T17:53:20.076663+0000","last_undegraded":"2024-05-03T17:53:20.076663+0000","last_fullsized":"2024-05-03T17:53:20.076663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:25.545290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077849+0000","last_change":"2024-05-03T17:52:32.690043+0000","last_active":"2024-05-03T17:53:20.077849+0000","last_peered":"2024-05-03T17:53:20.077849+0000","last_clean":"2024-05-03T17:53:20.077849+0000","last_became_active":"2024-05-03T17:52:32.689562+0000","last_became_peered":"2024-05-03T17:52:32.689562+0000","last_unstale":"2024-05-03T17:53:20.077849+0000","last_undegraded":"2024-05-03T17:53:20.077849+0000","last_fullsized":"2024-05-03T17:53:20.077849+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:59.014968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":97,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077303+0000","last_change":"2024-05-03T17:51:02.340854+0000","last_active":"2024-05-03T17:53:20.077303+0000","last_peered":"2024-05-03T17:53:20.077303+0000","last_clean":"2024-05-03T17:53:20.077303+0000","last_became_active":"2024-05-03T17:51:02.336454+0000","last_became_peered":"2024-05-03T17:51:02.336454+0000","last_unstale":"2024-05-03T17:53:20.077303+0000","last_undegraded":"2024-05-03T17:53:20.077303+0000","last_fullsized":"2024-05-03T17:53:20.077303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_clean_scrub_stamp":"2024-05-03T17:47:41.306771+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:18:08.512098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.887126+0000","last_change":"2024-05-03T17:53:19.887126+0000","last_active":"2024-05-03T17:53:19.887126+0000","last_peered":"2024-05-03T17:53:19.887126+0000","last_clean":"2024-05-03T17:53:19.887126+0000","last_became_active":"2024-05-03T17:53:19.886736+0000","last_became_peered":"2024-05-03T17:53:19.886736+0000","last_unstale":"2024-05-03T17:53:19.887126+0000","last_undegraded":"2024-05-03T17:53:19.887126+0000","last_fullsized":"2024-05-03T17:53:19.887126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:35:02.036825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077402+0000","last_change":"2024-05-03T17:52:32.690093+0000","last_active":"2024-05-03T17:53:20.077402+0000","last_peered":"2024-05-03T17:53:20.077402+0000","last_clean":"2024-05-03T17:53:20.077402+0000","last_became_active":"2024-05-03T17:52:32.689777+0000","last_became_peered":"2024-05-03T17:52:32.689777+0000","last_unstale":"2024-05-03T17:53:20.077402+0000","last_undegraded":"2024-05-03T17:53:20.077402+0000","last_fullsized":"2024-05-03T17:53:20.077402+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:09.693011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871341+0000","last_change":"2024-05-03T17:52:32.935589+0000","last_active":"2024-05-03T17:53:19.871341+0000","last_peered":"2024-05-03T17:53:19.871341+0000","last_clean":"2024-05-03T17:53:19.871341+0000","last_became_active":"2024-05-03T17:52:32.935221+0000","last_became_peered":"2024-05-03T17:52:32.935221+0000","last_unstale":"2024-05-03T17:53:19.871341+0000","last_undegraded":"2024-05-03T17:53:19.871341+0000","last_fullsized":"2024-05-03T17:53:19.871341+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:26:08.001761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867191+0000","last_change":"2024-05-03T17:52:35.724293+0000","last_active":"2024-05-03T17:53:19.867191+0000","last_peered":"2024-05-03T17:53:19.867191+0000","last_clean":"2024-05-03T17:53:19.867191+0000","last_became_active":"2024-05-03T17:52:35.723978+0000","last_became_peered":"2024-05-03T17:52:35.723978+0000","last_unstale":"2024-05-03T17:53:19.867191+0000","last_undegraded":"2024-05-03T17:53:19.867191+0000","last_fullsized":"2024-05-03T17:53:19.867191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:23.357872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869626+0000","last_change":"2024-05-03T17:52:32.687303+0000","last_active":"2024-05-03T17:53:19.869626+0000","last_peered":"2024-05-03T17:53:19.869626+0000","last_clean":"2024-05-03T17:53:19.869626+0000","last_became_active":"2024-05-03T17:52:32.686827+0000","last_became_peered":"2024-05-03T17:52:32.686827+0000","last_unstale":"2024-05-03T17:53:19.869626+0000","last_undegraded":"2024-05-03T17:53:19.869626+0000","last_fullsized":"2024-05-03T17:53:19.869626+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:11:53.565215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076856+0000","last_change":"2024-05-03T17:52:35.720898+0000","last_active":"2024-05-03T17:53:20.076856+0000","last_peered":"2024-05-03T17:53:20.076856+0000","last_clean":"2024-05-03T17:53:20.076856+0000","last_became_active":"2024-05-03T17:52:35.720572+0000","last_became_peered":"2024-05-03T17:52:35.720572+0000","last_unstale":"2024-05-03T17:53:20.076856+0000","last_undegraded":"2024-05-03T17:53:20.076856+0000","last_fullsized":"2024-05-03T17:53:20.076856+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:01.697167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077357+0000","last_change":"2024-05-03T17:52:33.249970+0000","last_active":"2024-05-03T17:53:20.077357+0000","last_peered":"2024-05-03T17:53:20.077357+0000","last_clean":"2024-05-03T17:53:20.077357+0000","last_became_active":"2024-05-03T17:52:33.249596+0000","last_became_peered":"2024-05-03T17:52:33.249596+0000","last_unstale":"2024-05-03T17:53:20.077357+0000","last_undegraded":"2024-05-03T17:53:20.077357+0000","last_fullsized":"2024-05-03T17:53:20.077357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:46:53.058586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870807+0000","last_change":"2024-05-03T17:52:35.722129+0000","last_active":"2024-05-03T17:53:19.870807+0000","last_peered":"2024-05-03T17:53:19.870807+0000","last_clean":"2024-05-03T17:53:19.870807+0000","last_became_active":"2024-05-03T17:52:35.721815+0000","last_became_peered":"2024-05-03T17:52:35.721815+0000","last_unstale":"2024-05-03T17:53:19.870807+0000","last_undegraded":"2024-05-03T17:53:19.870807+0000","last_fullsized":"2024-05-03T17:53:19.870807+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:33.781350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867376+0000","last_change":"2024-05-03T17:52:32.936157+0000","last_active":"2024-05-03T17:53:19.867376+0000","last_peered":"2024-05-03T17:53:19.867376+0000","last_clean":"2024-05-03T17:53:19.867376+0000","last_became_active":"2024-05-03T17:52:32.935842+0000","last_became_peered":"2024-05-03T17:52:32.935842+0000","last_unstale":"2024-05-03T17:53:19.867376+0000","last_undegraded":"2024-05-03T17:53:19.867376+0000","last_fullsized":"2024-05-03T17:53:19.867376+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:18.006451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870494+0000","last_change":"2024-05-03T17:52:35.735993+0000","last_active":"2024-05-03T17:53:19.870494+0000","last_peered":"2024-05-03T17:53:19.870494+0000","last_clean":"2024-05-03T17:53:19.870494+0000","last_became_active":"2024-05-03T17:52:35.735612+0000","last_became_peered":"2024-05-03T17:52:35.735612+0000","last_unstale":"2024-05-03T17:53:19.870494+0000","last_undegraded":"2024-05-03T17:53:19.870494+0000","last_fullsized":"2024-05-03T17:53:19.870494+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:59.216022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076573+0000","last_change":"2024-05-03T17:52:32.687147+0000","last_active":"2024-05-03T17:53:20.076573+0000","last_peered":"2024-05-03T17:53:20.076573+0000","last_clean":"2024-05-03T17:53:20.076573+0000","last_became_active":"2024-05-03T17:52:32.686831+0000","last_became_peered":"2024-05-03T17:52:32.686831+0000","last_unstale":"2024-05-03T17:53:20.076573+0000","last_undegraded":"2024-05-03T17:53:20.076573+0000","last_fullsized":"2024-05-03T17:53:20.076573+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:24:30.092829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869678+0000","last_change":"2024-05-03T17:52:35.721862+0000","last_active":"2024-05-03T17:53:19.869678+0000","last_peered":"2024-05-03T17:53:19.869678+0000","last_clean":"2024-05-03T17:53:19.869678+0000","last_became_active":"2024-05-03T17:52:35.721393+0000","last_became_peered":"2024-05-03T17:52:35.721393+0000","last_unstale":"2024-05-03T17:53:19.869678+0000","last_undegraded":"2024-05-03T17:53:19.869678+0000","last_fullsized":"2024-05-03T17:53:19.869678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:47:32.014817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885948+0000","last_change":"2024-05-03T17:52:32.687867+0000","last_active":"2024-05-03T17:53:19.885948+0000","last_peered":"2024-05-03T17:53:19.885948+0000","last_clean":"2024-05-03T17:53:19.885948+0000","last_became_active":"2024-05-03T17:52:32.687557+0000","last_became_peered":"2024-05-03T17:52:32.687557+0000","last_unstale":"2024-05-03T17:53:19.885948+0000","last_undegraded":"2024-05-03T17:53:19.885948+0000","last_fullsized":"2024-05-03T17:53:19.885948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:45.863180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076840+0000","last_change":"2024-05-03T17:52:35.731057+0000","last_active":"2024-05-03T17:53:20.076840+0000","last_peered":"2024-05-03T17:53:20.076840+0000","last_clean":"2024-05-03T17:53:20.076840+0000","last_became_active":"2024-05-03T17:52:35.729293+0000","last_became_peered":"2024-05-03T17:52:35.729293+0000","last_unstale":"2024-05-03T17:53:20.076840+0000","last_undegraded":"2024-05-03T17:53:20.076840+0000","last_fullsized":"2024-05-03T17:53:20.076840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:12:53.888078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870721+0000","last_change":"2024-05-03T17:52:32.689675+0000","last_active":"2024-05-03T17:53:19.870721+0000","last_peered":"2024-05-03T17:53:19.870721+0000","last_clean":"2024-05-03T17:53:19.870721+0000","last_became_active":"2024-05-03T17:52:32.689340+0000","last_became_peered":"2024-05-03T17:52:32.689340+0000","last_unstale":"2024-05-03T17:53:19.870721+0000","last_undegraded":"2024-05-03T17:53:19.870721+0000","last_fullsized":"2024-05-03T17:53:19.870721+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:31.398045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077600+0000","last_change":"2024-05-03T17:52:35.728228+0000","last_active":"2024-05-03T17:53:20.077600+0000","last_peered":"2024-05-03T17:53:20.077600+0000","last_clean":"2024-05-03T17:53:20.077600+0000","last_became_active":"2024-05-03T17:52:35.727233+0000","last_became_peered":"2024-05-03T17:52:35.727233+0000","last_unstale":"2024-05-03T17:53:20.077600+0000","last_undegraded":"2024-05-03T17:53:20.077600+0000","last_fullsized":"2024-05-03T17:53:20.077600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:57.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886416+0000","last_change":"2024-05-03T17:52:33.249690+0000","last_active":"2024-05-03T17:53:19.886416+0000","last_peered":"2024-05-03T17:53:19.886416+0000","last_clean":"2024-05-03T17:53:19.886416+0000","last_became_active":"2024-05-03T17:52:33.249302+0000","last_became_peered":"2024-05-03T17:52:33.249302+0000","last_unstale":"2024-05-03T17:53:19.886416+0000","last_undegraded":"2024-05-03T17:53:19.886416+0000","last_fullsized":"2024-05-03T17:53:19.886416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:05:48.700670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077737+0000","last_change":"2024-05-03T17:52:35.727171+0000","last_active":"2024-05-03T17:53:20.077737+0000","last_peered":"2024-05-03T17:53:20.077737+0000","last_clean":"2024-05-03T17:53:20.077737+0000","last_became_active":"2024-05-03T17:52:35.726834+0000","last_became_peered":"2024-05-03T17:52:35.726834+0000","last_unstale":"2024-05-03T17:53:20.077737+0000","last_undegraded":"2024-05-03T17:53:20.077737+0000","last_fullsized":"2024-05-03T17:53:20.077737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:40:09.938183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872806+0000","last_change":"2024-05-03T17:52:32.693694+0000","last_active":"2024-05-03T17:53:19.872806+0000","last_peered":"2024-05-03T17:53:19.872806+0000","last_clean":"2024-05-03T17:53:19.872806+0000","last_became_active":"2024-05-03T17:52:32.693310+0000","last_became_peered":"2024-05-03T17:52:32.693310+0000","last_unstale":"2024-05-03T17:53:19.872806+0000","last_undegraded":"2024-05-03T17:53:19.872806+0000","last_fullsized":"2024-05-03T17:53:19.872806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:59:37.449434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077538+0000","last_change":"2024-05-03T17:52:35.728267+0000","last_active":"2024-05-03T17:53:20.077538+0000","last_peered":"2024-05-03T17:53:20.077538+0000","last_clean":"2024-05-03T17:53:20.077538+0000","last_became_active":"2024-05-03T17:52:35.727902+0000","last_became_peered":"2024-05-03T17:52:35.727902+0000","last_unstale":"2024-05-03T17:53:20.077538+0000","last_undegraded":"2024-05-03T17:53:20.077538+0000","last_fullsized":"2024-05-03T17:53:20.077538+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:09.223850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867146+0000","last_change":"2024-05-03T17:52:32.687248+0000","last_active":"2024-05-03T17:53:19.867146+0000","last_peered":"2024-05-03T17:53:19.867146+0000","last_clean":"2024-05-03T17:53:19.867146+0000","last_became_active":"2024-05-03T17:52:32.686921+0000","last_became_peered":"2024-05-03T17:52:32.686921+0000","last_unstale":"2024-05-03T17:53:19.867146+0000","last_undegraded":"2024-05-03T17:53:19.867146+0000","last_fullsized":"2024-05-03T17:53:19.867146+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:35:40.849484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077858+0000","last_change":"2024-05-03T17:52:35.729383+0000","last_active":"2024-05-03T17:53:20.077858+0000","last_peered":"2024-05-03T17:53:20.077858+0000","last_clean":"2024-05-03T17:53:20.077858+0000","last_became_active":"2024-05-03T17:52:35.729037+0000","last_became_peered":"2024-05-03T17:52:35.729037+0000","last_unstale":"2024-05-03T17:53:20.077858+0000","last_undegraded":"2024-05-03T17:53:20.077858+0000","last_fullsized":"2024-05-03T17:53:20.077858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:03:03.815770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869741+0000","last_change":"2024-05-03T17:52:35.731809+0000","last_active":"2024-05-03T17:53:19.869741+0000","last_peered":"2024-05-03T17:53:19.869741+0000","last_clean":"2024-05-03T17:53:19.869741+0000","last_became_active":"2024-05-03T17:52:35.729360+0000","last_became_peered":"2024-05-03T17:52:35.729360+0000","last_unstale":"2024-05-03T17:53:19.869741+0000","last_undegraded":"2024-05-03T17:53:19.869741+0000","last_fullsized":"2024-05-03T17:53:19.869741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:22.181184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886168+0000","last_change":"2024-05-03T17:52:35.727771+0000","last_active":"2024-05-03T17:53:19.886168+0000","last_peered":"2024-05-03T17:53:19.886168+0000","last_clean":"2024-05-03T17:53:19.886168+0000","last_became_active":"2024-05-03T17:52:35.727414+0000","last_became_peered":"2024-05-03T17:52:35.727414+0000","last_unstale":"2024-05-03T17:53:19.886168+0000","last_undegraded":"2024-05-03T17:53:19.886168+0000","last_fullsized":"2024-05-03T17:53:19.886168+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:09:02.905481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885867+0000","last_change":"2024-05-03T17:52:35.727433+0000","last_active":"2024-05-03T17:53:19.885867+0000","last_peered":"2024-05-03T17:53:19.885867+0000","last_clean":"2024-05-03T17:53:19.885867+0000","last_became_active":"2024-05-03T17:52:35.727123+0000","last_became_peered":"2024-05-03T17:52:35.727123+0000","last_unstale":"2024-05-03T17:53:19.885867+0000","last_undegraded":"2024-05-03T17:53:19.885867+0000","last_fullsized":"2024-05-03T17:53:19.885867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:41:36.871788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870666+0000","last_change":"2024-05-03T17:52:35.721332+0000","last_active":"2024-05-03T17:53:19.870666+0000","last_peered":"2024-05-03T17:53:19.870666+0000","last_clean":"2024-05-03T17:53:19.870666+0000","last_became_active":"2024-05-03T17:52:35.721004+0000","last_became_peered":"2024-05-03T17:52:35.721004+0000","last_unstale":"2024-05-03T17:53:19.870666+0000","last_undegraded":"2024-05-03T17:53:19.870666+0000","last_fullsized":"2024-05-03T17:53:19.870666+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:35:03.443163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867305+0000","last_change":"2024-05-03T17:52:35.723649+0000","last_active":"2024-05-03T17:53:19.867305+0000","last_peered":"2024-05-03T17:53:19.867305+0000","last_clean":"2024-05-03T17:53:19.867305+0000","last_became_active":"2024-05-03T17:52:35.723303+0000","last_became_peered":"2024-05-03T17:52:35.723303+0000","last_unstale":"2024-05-03T17:53:19.867305+0000","last_undegraded":"2024-05-03T17:53:19.867305+0000","last_fullsized":"2024-05-03T17:53:19.867305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:02:49.093039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870096+0000","last_change":"2024-05-03T17:52:35.732505+0000","last_active":"2024-05-03T17:53:19.870096+0000","last_peered":"2024-05-03T17:53:19.870096+0000","last_clean":"2024-05-03T17:53:19.870096+0000","last_became_active":"2024-05-03T17:52:35.732180+0000","last_became_peered":"2024-05-03T17:52:35.732180+0000","last_unstale":"2024-05-03T17:53:19.870096+0000","last_undegraded":"2024-05-03T17:53:19.870096+0000","last_fullsized":"2024-05-03T17:53:19.870096+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.371260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886349+0000","last_change":"2024-05-03T17:52:35.732217+0000","last_active":"2024-05-03T17:53:19.886349+0000","last_peered":"2024-05-03T17:53:19.886349+0000","last_clean":"2024-05-03T17:53:19.886349+0000","last_became_active":"2024-05-03T17:52:35.731868+0000","last_became_peered":"2024-05-03T17:52:35.731868+0000","last_unstale":"2024-05-03T17:53:19.886349+0000","last_undegraded":"2024-05-03T17:53:19.886349+0000","last_fullsized":"2024-05-03T17:53:19.886349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:04:17.521528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076813+0000","last_change":"2024-05-03T17:52:35.723641+0000","last_active":"2024-05-03T17:53:20.076813+0000","last_peered":"2024-05-03T17:53:20.076813+0000","last_clean":"2024-05-03T17:53:20.076813+0000","last_became_active":"2024-05-03T17:52:35.723323+0000","last_became_peered":"2024-05-03T17:52:35.723323+0000","last_unstale":"2024-05-03T17:53:20.076813+0000","last_undegraded":"2024-05-03T17:53:20.076813+0000","last_fullsized":"2024-05-03T17:53:20.076813+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:45:34.032407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872477+0000","last_change":"2024-05-03T17:52:35.731008+0000","last_active":"2024-05-03T17:53:19.872477+0000","last_peered":"2024-05-03T17:53:19.872477+0000","last_clean":"2024-05-03T17:53:19.872477+0000","last_became_active":"2024-05-03T17:52:35.730627+0000","last_became_peered":"2024-05-03T17:52:35.730627+0000","last_unstale":"2024-05-03T17:53:19.872477+0000","last_undegraded":"2024-05-03T17:53:19.872477+0000","last_fullsized":"2024-05-03T17:53:19.872477+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:30:32.884518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076519+0000","last_change":"2024-05-03T17:52:35.726938+0000","last_active":"2024-05-03T17:53:20.076519+0000","last_peered":"2024-05-03T17:53:20.076519+0000","last_clean":"2024-05-03T17:53:20.076519+0000","last_became_active":"2024-05-03T17:52:35.726611+0000","last_became_peered":"2024-05-03T17:52:35.726611+0000","last_unstale":"2024-05-03T17:53:20.076519+0000","last_undegraded":"2024-05-03T17:53:20.076519+0000","last_fullsized":"2024-05-03T17:53:20.076519+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:11.587035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867632+0000","last_change":"2024-05-03T17:52:35.722562+0000","last_active":"2024-05-03T17:53:19.867632+0000","last_peered":"2024-05-03T17:53:19.867632+0000","last_clean":"2024-05-03T17:53:19.867632+0000","last_became_active":"2024-05-03T17:52:35.722229+0000","last_became_peered":"2024-05-03T17:52:35.722229+0000","last_unstale":"2024-05-03T17:53:19.867632+0000","last_undegraded":"2024-05-03T17:53:19.867632+0000","last_fullsized":"2024-05-03T17:53:19.867632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:57:06.730533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867188+0000","last_change":"2024-05-03T17:52:35.722707+0000","last_active":"2024-05-03T17:53:19.867188+0000","last_peered":"2024-05-03T17:53:19.867188+0000","last_clean":"2024-05-03T17:53:19.867188+0000","last_became_active":"2024-05-03T17:52:35.721847+0000","last_became_peered":"2024-05-03T17:52:35.721847+0000","last_unstale":"2024-05-03T17:53:19.867188+0000","last_undegraded":"2024-05-03T17:53:19.867188+0000","last_fullsized":"2024-05-03T17:53:19.867188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:56:21.620074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870547+0000","last_change":"2024-05-03T17:52:35.731810+0000","last_active":"2024-05-03T17:53:19.870547+0000","last_peered":"2024-05-03T17:53:19.870547+0000","last_clean":"2024-05-03T17:53:19.870547+0000","last_became_active":"2024-05-03T17:52:35.728986+0000","last_became_peered":"2024-05-03T17:52:35.728986+0000","last_unstale":"2024-05-03T17:53:19.870547+0000","last_undegraded":"2024-05-03T17:53:19.870547+0000","last_fullsized":"2024-05-03T17:53:19.870547+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:18.275792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280811,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59209,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":253,"num_read_kb":190,"num_write":161,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":175975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743451,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":417972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906977,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1064661,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070502,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":418208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266730,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":418128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":1010449,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626484,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":417597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":470555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":418868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":472296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":872448,"data_stored":417908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":860160,"data_stored":418425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":1009742,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T17:53:34.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:34 smithi149 ceph-mon[33409]: pgmap v316: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:34.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:34 smithi149 ceph-mon[33409]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:34.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:34 smithi192 ceph-mon[33753]: pgmap v316: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:34.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:34 smithi192 ceph-mon[33753]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:34.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:34 smithi122 ceph-mon[29744]: pgmap v316: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:34.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:34 smithi122 ceph-mon[29744]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:36.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:36 smithi149 ceph-mon[33409]: pgmap v317: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:36.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:36 smithi192 ceph-mon[33753]: pgmap v317: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:36.674 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T17:53:36.700 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:36 smithi122 ceph-mon[29744]: pgmap v317: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T17:53:36.941 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:38.351 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:38 smithi122 ceph-mon[29744]: pgmap v318: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-03T17:53:38.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:38 smithi149 ceph-mon[33409]: pgmap v318: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-03T17:53:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.592+0000 7f7644d31700 1 -- 172.21.15.122:0/4893761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7640108410 msgr2=0x7f76401087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.592+0000 7f7644d31700 1 --2- 172.21.15.122:0/4893761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7640108410 0x7f76401087f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7630009990 tx=0x7f7630009ca0 comp rx=0 tx=0).stop 2024-05-03T17:53:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.593+0000 7f7644d31700 1 -- 172.21.15.122:0/4893761 shutdown_connections 2024-05-03T17:53:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.593+0000 7f7644d31700 1 --2- 172.21.15.122:0/4893761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7640102d60 0x7f764010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.593+0000 7f7644d31700 1 --2- 172.21.15.122:0/4893761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7640108410 0x7f76401087f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.593+0000 7f7644d31700 1 --2- 172.21.15.122:0/4893761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76401023a0 0x7f7640102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.593+0000 7f7644d31700 1 -- 172.21.15.122:0/4893761 >> 172.21.15.122:0/4893761 conn(0x7f7640079350 msgr2=0x7f7640079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.593+0000 7f7644d31700 1 -- 172.21.15.122:0/4893761 shutdown_connections 2024-05-03T17:53:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.593+0000 7f7644d31700 1 -- 172.21.15.122:0/4893761 wait complete. 2024-05-03T17:53:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.593+0000 7f7644d31700 1 Processor -- start 2024-05-03T17:53:38.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f7644d31700 1 -- start start 2024-05-03T17:53:38.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f7644d31700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76401023a0 0x7f7640199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:38.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f7644d31700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7640102d60 0x7f764019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:38.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f7644d31700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7640108410 0x7f764019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:38.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f7644d31700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7640112680 con 0x7f76401023a0 2024-05-03T17:53:38.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f7644d31700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7640112500 con 0x7f7640102d60 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f7644d31700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7640112800 con 0x7f7640108410 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f763e59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76401023a0 0x7f7640199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f763e59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76401023a0 0x7f7640199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48458/0 (socket says 172.21.15.122:48458) 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f763e59c700 1 -- 172.21.15.122:0/2365550232 learned_addr learned my addr 172.21.15.122:0/2365550232 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f763ed9d700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7640108410 0x7f764019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f7635d9b700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7640102d60 0x7f764019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f763e59c700 1 -- 172.21.15.122:0/2365550232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7640108410 msgr2=0x7f764019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f763e59c700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7640108410 0x7f764019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f763e59c700 1 -- 172.21.15.122:0/2365550232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7640102d60 msgr2=0x7f764019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f763e59c700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7640102d60 0x7f764019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f763e59c700 1 -- 172.21.15.122:0/2365550232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764019ed50 con 0x7f76401023a0 2024-05-03T17:53:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f7635d9b700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7640102d60 0x7f764019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:53:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.594+0000 7f763ed9d700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7640108410 0x7f764019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:53:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.595+0000 7f763e59c700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76401023a0 0x7f7640199d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7630000c00 tx=0x7f76300057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.595+0000 7f7637fff700 1 -- 172.21.15.122:0/2365550232 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630027070 con 0x7f76401023a0 2024-05-03T17:53:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.595+0000 7f7637fff700 1 -- 172.21.15.122:0/2365550232 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7630004020 con 0x7f76401023a0 2024-05-03T17:53:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.595+0000 7f7644d31700 1 -- 172.21.15.122:0/2365550232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764019efe0 con 0x7f76401023a0 2024-05-03T17:53:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.595+0000 7f7637fff700 1 -- 172.21.15.122:0/2365550232 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630022e00 con 0x7f76401023a0 2024-05-03T17:53:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.595+0000 7f7644d31700 1 -- 172.21.15.122:0/2365550232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764006cec0 con 0x7f76401023a0 2024-05-03T17:53:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.596+0000 7f7644d31700 1 -- 172.21.15.122:0/2365550232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7640103ac0 con 0x7f76401023a0 2024-05-03T17:53:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.597+0000 7f7637fff700 1 -- 172.21.15.122:0/2365550232 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f763002a070 con 0x7f76401023a0 2024-05-03T17:53:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.598+0000 7f7637fff700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f762c0b2910 0x7f762c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.598+0000 7f7635d9b700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f762c0b2910 0x7f762c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.598+0000 7f7637fff700 1 -- 172.21.15.122:0/2365550232 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f7630005030 con 0x7f76401023a0 2024-05-03T17:53:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.598+0000 7f7635d9b700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f762c0b2910 0x7f762c0b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f76400798e0 tx=0x7f7628008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:38.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.601+0000 7f7637fff700 1 -- 172.21.15.122:0/2365550232 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7630031020 con 0x7f76401023a0 2024-05-03T17:53:38.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:38 smithi192 ceph-mon[33753]: pgmap v318: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-03T17:53:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.803+0000 7f7644d31700 1 -- 172.21.15.122:0/2365550232 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f764019b0a0 con 0x7f762c0b2910 2024-05-03T17:53:38.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.810+0000 7f7637fff700 1 -- 172.21.15.122:0/2365550232 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258859 (secure 0 0 0) 0x7f764019b0a0 con 0x7f762c0b2910 2024-05-03T17:53:38.809 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.813+0000 7f7644d31700 1 -- 172.21.15.122:0/2365550232 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f762c0b2910 msgr2=0x7f762c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.813+0000 7f7644d31700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f762c0b2910 0x7f762c0b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f76400798e0 tx=0x7f7628008040 comp rx=0 tx=0).stop 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.813+0000 7f7644d31700 1 -- 172.21.15.122:0/2365550232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76401023a0 msgr2=0x7f7640199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.813+0000 7f7644d31700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76401023a0 0x7f7640199d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7630000c00 tx=0x7f76300057c0 comp rx=0 tx=0).stop 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.813+0000 7f7644d31700 1 -- 172.21.15.122:0/2365550232 shutdown_connections 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.813+0000 7f7644d31700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f762c0b2910 0x7f762c0b4dd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.813+0000 7f7644d31700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7640108410 0x7f764019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.813+0000 7f7644d31700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76401023a0 0x7f7640199d40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.813+0000 7f7644d31700 1 --2- 172.21.15.122:0/2365550232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7640102d60 0x7f764019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.813+0000 7f7644d31700 1 -- 172.21.15.122:0/2365550232 >> 172.21.15.122:0/2365550232 conn(0x7f7640079350 msgr2=0x7f764006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.814+0000 7f7644d31700 1 -- 172.21.15.122:0/2365550232 shutdown_connections 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:38.814+0000 7f7644d31700 1 -- 172.21.15.122:0/2365550232 wait complete. 2024-05-03T17:53:38.811 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-05-03T17:53:39.448 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-05-03T17:53:38.552864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649811,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":364,"num_read_kb":274,"num_write":286,"num_write_kb":2257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376344,"kb_used_data":10500,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565480,"statfs":{"total":1151940427776,"available":1151555051520,"internally_reserved":0,"allocated":10752000,"data_stored":6958113,"data_compressed":107130,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":47,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003024"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870114+0000","last_change":"2024-05-03T17:52:35.725057+0000","last_active":"2024-05-03T17:53:19.870114+0000","last_peered":"2024-05-03T17:53:19.870114+0000","last_clean":"2024-05-03T17:53:19.870114+0000","last_became_active":"2024-05-03T17:52:35.724632+0000","last_became_peered":"2024-05-03T17:52:35.724632+0000","last_unstale":"2024-05-03T17:53:19.870114+0000","last_undegraded":"2024-05-03T17:53:19.870114+0000","last_fullsized":"2024-05-03T17:53:19.870114+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:46.065357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870650+0000","last_change":"2024-05-03T17:52:35.728500+0000","last_active":"2024-05-03T17:53:19.870650+0000","last_peered":"2024-05-03T17:53:19.870650+0000","last_clean":"2024-05-03T17:53:19.870650+0000","last_became_active":"2024-05-03T17:52:35.728210+0000","last_became_peered":"2024-05-03T17:52:35.728210+0000","last_unstale":"2024-05-03T17:53:19.870650+0000","last_undegraded":"2024-05-03T17:53:19.870650+0000","last_fullsized":"2024-05-03T17:53:19.870650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:33:23.410617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870644+0000","last_change":"2024-05-03T17:52:35.726300+0000","last_active":"2024-05-03T17:53:19.870644+0000","last_peered":"2024-05-03T17:53:19.870644+0000","last_clean":"2024-05-03T17:53:19.870644+0000","last_became_active":"2024-05-03T17:52:35.725994+0000","last_became_peered":"2024-05-03T17:52:35.725994+0000","last_unstale":"2024-05-03T17:53:19.870644+0000","last_undegraded":"2024-05-03T17:53:19.870644+0000","last_fullsized":"2024-05-03T17:53:19.870644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:31:19.723453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077278+0000","last_change":"2024-05-03T17:52:35.728105+0000","last_active":"2024-05-03T17:53:20.077278+0000","last_peered":"2024-05-03T17:53:20.077278+0000","last_clean":"2024-05-03T17:53:20.077278+0000","last_became_active":"2024-05-03T17:52:35.727762+0000","last_became_peered":"2024-05-03T17:52:35.727762+0000","last_unstale":"2024-05-03T17:53:20.077278+0000","last_undegraded":"2024-05-03T17:53:20.077278+0000","last_fullsized":"2024-05-03T17:53:20.077278+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.640057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870597+0000","last_change":"2024-05-03T17:52:35.734516+0000","last_active":"2024-05-03T17:53:19.870597+0000","last_peered":"2024-05-03T17:53:19.870597+0000","last_clean":"2024-05-03T17:53:19.870597+0000","last_became_active":"2024-05-03T17:52:35.734125+0000","last_became_peered":"2024-05-03T17:52:35.734125+0000","last_unstale":"2024-05-03T17:53:19.870597+0000","last_undegraded":"2024-05-03T17:53:19.870597+0000","last_fullsized":"2024-05-03T17:53:19.870597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:39:25.103137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871539+0000","last_change":"2024-05-03T17:52:35.723427+0000","last_active":"2024-05-03T17:53:19.871539+0000","last_peered":"2024-05-03T17:53:19.871539+0000","last_clean":"2024-05-03T17:53:19.871539+0000","last_became_active":"2024-05-03T17:52:35.723113+0000","last_became_peered":"2024-05-03T17:52:35.723113+0000","last_unstale":"2024-05-03T17:53:19.871539+0000","last_undegraded":"2024-05-03T17:53:19.871539+0000","last_fullsized":"2024-05-03T17:53:19.871539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:37:14.606347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077441+0000","last_change":"2024-05-03T17:52:35.730858+0000","last_active":"2024-05-03T17:53:20.077441+0000","last_peered":"2024-05-03T17:53:20.077441+0000","last_clean":"2024-05-03T17:53:20.077441+0000","last_became_active":"2024-05-03T17:52:35.730453+0000","last_became_peered":"2024-05-03T17:52:35.730453+0000","last_unstale":"2024-05-03T17:53:20.077441+0000","last_undegraded":"2024-05-03T17:53:20.077441+0000","last_fullsized":"2024-05-03T17:53:20.077441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:21:17.803027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076565+0000","last_change":"2024-05-03T17:52:35.730547+0000","last_active":"2024-05-03T17:53:20.076565+0000","last_peered":"2024-05-03T17:53:20.076565+0000","last_clean":"2024-05-03T17:53:20.076565+0000","last_became_active":"2024-05-03T17:52:35.729582+0000","last_became_peered":"2024-05-03T17:52:35.729582+0000","last_unstale":"2024-05-03T17:53:20.076565+0000","last_undegraded":"2024-05-03T17:53:20.076565+0000","last_fullsized":"2024-05-03T17:53:20.076565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:24:55.110537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.868010+0000","last_change":"2024-05-03T17:52:35.726305+0000","last_active":"2024-05-03T17:53:19.868010+0000","last_peered":"2024-05-03T17:53:19.868010+0000","last_clean":"2024-05-03T17:53:19.868010+0000","last_became_active":"2024-05-03T17:52:35.725858+0000","last_became_peered":"2024-05-03T17:52:35.725858+0000","last_unstale":"2024-05-03T17:53:19.868010+0000","last_undegraded":"2024-05-03T17:53:19.868010+0000","last_fullsized":"2024-05-03T17:53:19.868010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:59.347307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871846+0000","last_change":"2024-05-03T17:52:35.726865+0000","last_active":"2024-05-03T17:53:19.871846+0000","last_peered":"2024-05-03T17:53:19.871846+0000","last_clean":"2024-05-03T17:53:19.871846+0000","last_became_active":"2024-05-03T17:52:35.726483+0000","last_became_peered":"2024-05-03T17:52:35.726483+0000","last_unstale":"2024-05-03T17:53:19.871846+0000","last_undegraded":"2024-05-03T17:53:19.871846+0000","last_fullsized":"2024-05-03T17:53:19.871846+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:17.851133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077159+0000","last_change":"2024-05-03T17:52:35.730551+0000","last_active":"2024-05-03T17:53:20.077159+0000","last_peered":"2024-05-03T17:53:20.077159+0000","last_clean":"2024-05-03T17:53:20.077159+0000","last_became_active":"2024-05-03T17:52:35.730187+0000","last_became_peered":"2024-05-03T17:52:35.730187+0000","last_unstale":"2024-05-03T17:53:20.077159+0000","last_undegraded":"2024-05-03T17:53:20.077159+0000","last_fullsized":"2024-05-03T17:53:20.077159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:18:56.211061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077071+0000","last_change":"2024-05-03T17:52:35.730040+0000","last_active":"2024-05-03T17:53:20.077071+0000","last_peered":"2024-05-03T17:53:20.077071+0000","last_clean":"2024-05-03T17:53:20.077071+0000","last_became_active":"2024-05-03T17:52:35.729549+0000","last_became_peered":"2024-05-03T17:52:35.729549+0000","last_unstale":"2024-05-03T17:53:20.077071+0000","last_undegraded":"2024-05-03T17:53:20.077071+0000","last_fullsized":"2024-05-03T17:53:20.077071+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:50:05.052247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867289+0000","last_change":"2024-05-03T17:52:35.726709+0000","last_active":"2024-05-03T17:53:19.867289+0000","last_peered":"2024-05-03T17:53:19.867289+0000","last_clean":"2024-05-03T17:53:19.867289+0000","last_became_active":"2024-05-03T17:52:35.726166+0000","last_became_peered":"2024-05-03T17:52:35.726166+0000","last_unstale":"2024-05-03T17:53:19.867289+0000","last_undegraded":"2024-05-03T17:53:19.867289+0000","last_fullsized":"2024-05-03T17:53:19.867289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.164622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870439+0000","last_change":"2024-05-03T17:52:35.727583+0000","last_active":"2024-05-03T17:53:19.870439+0000","last_peered":"2024-05-03T17:53:19.870439+0000","last_clean":"2024-05-03T17:53:19.870439+0000","last_became_active":"2024-05-03T17:52:35.727131+0000","last_became_peered":"2024-05-03T17:52:35.727131+0000","last_unstale":"2024-05-03T17:53:19.870439+0000","last_undegraded":"2024-05-03T17:53:19.870439+0000","last_fullsized":"2024-05-03T17:53:19.870439+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:28:21.255954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.887563+0000","last_change":"2024-05-03T17:53:19.887563+0000","last_active":"2024-05-03T17:53:19.887563+0000","last_peered":"2024-05-03T17:53:19.887563+0000","last_clean":"2024-05-03T17:53:19.887563+0000","last_became_active":"2024-05-03T17:53:19.887068+0000","last_became_peered":"2024-05-03T17:53:19.887068+0000","last_unstale":"2024-05-03T17:53:19.887563+0000","last_undegraded":"2024-05-03T17:53:19.887563+0000","last_fullsized":"2024-05-03T17:53:19.887563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:49:49.354138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867360+0000","last_change":"2024-05-03T17:52:35.717354+0000","last_active":"2024-05-03T17:53:19.867360+0000","last_peered":"2024-05-03T17:53:19.867360+0000","last_clean":"2024-05-03T17:53:19.867360+0000","last_became_active":"2024-05-03T17:52:35.717025+0000","last_became_peered":"2024-05-03T17:52:35.717025+0000","last_unstale":"2024-05-03T17:53:19.867360+0000","last_undegraded":"2024-05-03T17:53:19.867360+0000","last_fullsized":"2024-05-03T17:53:19.867360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:11:07.778484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870387+0000","last_change":"2024-05-03T17:52:35.725626+0000","last_active":"2024-05-03T17:53:19.870387+0000","last_peered":"2024-05-03T17:53:19.870387+0000","last_clean":"2024-05-03T17:53:19.870387+0000","last_became_active":"2024-05-03T17:52:35.725239+0000","last_became_peered":"2024-05-03T17:52:35.725239+0000","last_unstale":"2024-05-03T17:53:19.870387+0000","last_undegraded":"2024-05-03T17:53:19.870387+0000","last_fullsized":"2024-05-03T17:53:19.870387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:44:18.359484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869622+0000","last_change":"2024-05-03T17:52:35.724250+0000","last_active":"2024-05-03T17:53:19.869622+0000","last_peered":"2024-05-03T17:53:19.869622+0000","last_clean":"2024-05-03T17:53:19.869622+0000","last_became_active":"2024-05-03T17:52:35.723998+0000","last_became_peered":"2024-05-03T17:52:35.723998+0000","last_unstale":"2024-05-03T17:53:19.869622+0000","last_undegraded":"2024-05-03T17:53:19.869622+0000","last_fullsized":"2024-05-03T17:53:19.869622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:03:32.939559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886262+0000","last_change":"2024-05-03T17:52:35.726237+0000","last_active":"2024-05-03T17:53:19.886262+0000","last_peered":"2024-05-03T17:53:19.886262+0000","last_clean":"2024-05-03T17:53:19.886262+0000","last_became_active":"2024-05-03T17:52:35.725847+0000","last_became_peered":"2024-05-03T17:52:35.725847+0000","last_unstale":"2024-05-03T17:53:19.886262+0000","last_undegraded":"2024-05-03T17:53:19.886262+0000","last_fullsized":"2024-05-03T17:53:19.886262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:06.472532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869723+0000","last_change":"2024-05-03T17:52:32.687198+0000","last_active":"2024-05-03T17:53:19.869723+0000","last_peered":"2024-05-03T17:53:19.869723+0000","last_clean":"2024-05-03T17:53:19.869723+0000","last_became_active":"2024-05-03T17:52:32.686848+0000","last_became_peered":"2024-05-03T17:52:32.686848+0000","last_unstale":"2024-05-03T17:53:19.869723+0000","last_undegraded":"2024-05-03T17:53:19.869723+0000","last_fullsized":"2024-05-03T17:53:19.869723+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:08:24.244371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076745+0000","last_change":"2024-05-03T17:52:35.730595+0000","last_active":"2024-05-03T17:53:20.076745+0000","last_peered":"2024-05-03T17:53:20.076745+0000","last_clean":"2024-05-03T17:53:20.076745+0000","last_became_active":"2024-05-03T17:52:35.729999+0000","last_became_peered":"2024-05-03T17:52:35.729999+0000","last_unstale":"2024-05-03T17:53:20.076745+0000","last_undegraded":"2024-05-03T17:53:20.076745+0000","last_fullsized":"2024-05-03T17:53:20.076745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:14.485756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076923+0000","last_change":"2024-05-03T17:52:32.935239+0000","last_active":"2024-05-03T17:53:20.076923+0000","last_peered":"2024-05-03T17:53:20.076923+0000","last_clean":"2024-05-03T17:53:20.076923+0000","last_became_active":"2024-05-03T17:52:32.934915+0000","last_became_peered":"2024-05-03T17:52:32.934915+0000","last_unstale":"2024-05-03T17:53:20.076923+0000","last_undegraded":"2024-05-03T17:53:20.076923+0000","last_fullsized":"2024-05-03T17:53:20.076923+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:20.785071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870934+0000","last_change":"2024-05-03T17:52:35.724940+0000","last_active":"2024-05-03T17:53:19.870934+0000","last_peered":"2024-05-03T17:53:19.870934+0000","last_clean":"2024-05-03T17:53:19.870934+0000","last_became_active":"2024-05-03T17:52:35.724603+0000","last_became_peered":"2024-05-03T17:52:35.724603+0000","last_unstale":"2024-05-03T17:53:19.870934+0000","last_undegraded":"2024-05-03T17:53:19.870934+0000","last_fullsized":"2024-05-03T17:53:19.870934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:43.640499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871451+0000","last_change":"2024-05-03T17:52:32.688202+0000","last_active":"2024-05-03T17:53:19.871451+0000","last_peered":"2024-05-03T17:53:19.871451+0000","last_clean":"2024-05-03T17:53:19.871451+0000","last_became_active":"2024-05-03T17:52:32.687893+0000","last_became_peered":"2024-05-03T17:52:32.687893+0000","last_unstale":"2024-05-03T17:53:19.871451+0000","last_undegraded":"2024-05-03T17:53:19.871451+0000","last_fullsized":"2024-05-03T17:53:19.871451+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:43.342832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867555+0000","last_change":"2024-05-03T17:52:35.720980+0000","last_active":"2024-05-03T17:53:19.867555+0000","last_peered":"2024-05-03T17:53:19.867555+0000","last_clean":"2024-05-03T17:53:19.867555+0000","last_became_active":"2024-05-03T17:52:35.720582+0000","last_became_peered":"2024-05-03T17:52:35.720582+0000","last_unstale":"2024-05-03T17:53:19.867555+0000","last_undegraded":"2024-05-03T17:53:19.867555+0000","last_fullsized":"2024-05-03T17:53:19.867555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:01:22.938186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.877543+0000","last_change":"2024-05-03T17:53:19.877543+0000","last_active":"2024-05-03T17:53:19.877543+0000","last_peered":"2024-05-03T17:53:19.877543+0000","last_clean":"2024-05-03T17:53:19.877543+0000","last_became_active":"2024-05-03T17:53:19.873988+0000","last_became_peered":"2024-05-03T17:53:19.873988+0000","last_unstale":"2024-05-03T17:53:19.877543+0000","last_undegraded":"2024-05-03T17:53:19.877543+0000","last_fullsized":"2024-05-03T17:53:19.877543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:49:49.105623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.079283+0000","last_change":"2024-05-03T17:53:20.078906+0000","last_active":"2024-05-03T17:53:20.079283+0000","last_peered":"2024-05-03T17:53:20.079283+0000","last_clean":"2024-05-03T17:53:20.079283+0000","last_became_active":"2024-05-03T17:53:20.078538+0000","last_became_peered":"2024-05-03T17:53:20.078538+0000","last_unstale":"2024-05-03T17:53:20.079283+0000","last_undegraded":"2024-05-03T17:53:20.079283+0000","last_fullsized":"2024-05-03T17:53:20.079283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:59:46.945417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":75,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:32.822933+0000","last_change":"2024-05-03T17:52:32.692735+0000","last_active":"2024-05-03T17:53:32.822933+0000","last_peered":"2024-05-03T17:53:32.822933+0000","last_clean":"2024-05-03T17:53:32.822933+0000","last_became_active":"2024-05-03T17:52:32.692394+0000","last_became_peered":"2024-05-03T17:52:32.692394+0000","last_unstale":"2024-05-03T17:53:32.822933+0000","last_undegraded":"2024-05-03T17:53:32.822933+0000","last_fullsized":"2024-05-03T17:53:32.822933+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:40:11.434751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870798+0000","last_change":"2024-05-03T17:52:35.727167+0000","last_active":"2024-05-03T17:53:19.870798+0000","last_peered":"2024-05-03T17:53:19.870798+0000","last_clean":"2024-05-03T17:53:19.870798+0000","last_became_active":"2024-05-03T17:52:35.726792+0000","last_became_peered":"2024-05-03T17:52:35.726792+0000","last_unstale":"2024-05-03T17:53:19.870798+0000","last_undegraded":"2024-05-03T17:53:19.870798+0000","last_fullsized":"2024-05-03T17:53:19.870798+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:54:01.823025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871452+0000","last_change":"2024-05-03T17:52:32.687573+0000","last_active":"2024-05-03T17:53:19.871452+0000","last_peered":"2024-05-03T17:53:19.871452+0000","last_clean":"2024-05-03T17:53:19.871452+0000","last_became_active":"2024-05-03T17:52:32.686158+0000","last_became_peered":"2024-05-03T17:52:32.686158+0000","last_unstale":"2024-05-03T17:53:19.871452+0000","last_undegraded":"2024-05-03T17:53:19.871452+0000","last_fullsized":"2024-05-03T17:53:19.871452+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:19.891479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873172+0000","last_change":"2024-05-03T17:52:35.728049+0000","last_active":"2024-05-03T17:53:19.873172+0000","last_peered":"2024-05-03T17:53:19.873172+0000","last_clean":"2024-05-03T17:53:19.873172+0000","last_became_active":"2024-05-03T17:52:35.727713+0000","last_became_peered":"2024-05-03T17:52:35.727713+0000","last_unstale":"2024-05-03T17:53:19.873172+0000","last_undegraded":"2024-05-03T17:53:19.873172+0000","last_fullsized":"2024-05-03T17:53:19.873172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:28:01.798457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867118+0000","last_change":"2024-05-03T17:52:32.685679+0000","last_active":"2024-05-03T17:53:19.867118+0000","last_peered":"2024-05-03T17:53:19.867118+0000","last_clean":"2024-05-03T17:53:19.867118+0000","last_became_active":"2024-05-03T17:52:32.685306+0000","last_became_peered":"2024-05-03T17:52:32.685306+0000","last_unstale":"2024-05-03T17:53:19.867118+0000","last_undegraded":"2024-05-03T17:53:19.867118+0000","last_fullsized":"2024-05-03T17:53:19.867118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:48:22.218310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076501+0000","last_change":"2024-05-03T17:52:35.717111+0000","last_active":"2024-05-03T17:53:20.076501+0000","last_peered":"2024-05-03T17:53:20.076501+0000","last_clean":"2024-05-03T17:53:20.076501+0000","last_became_active":"2024-05-03T17:52:35.716782+0000","last_became_peered":"2024-05-03T17:52:35.716782+0000","last_unstale":"2024-05-03T17:53:20.076501+0000","last_undegraded":"2024-05-03T17:53:20.076501+0000","last_fullsized":"2024-05-03T17:53:20.076501+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:31:57.766927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872477+0000","last_change":"2024-05-03T17:52:32.690846+0000","last_active":"2024-05-03T17:53:19.872477+0000","last_peered":"2024-05-03T17:53:19.872477+0000","last_clean":"2024-05-03T17:53:19.872477+0000","last_became_active":"2024-05-03T17:52:32.690506+0000","last_became_peered":"2024-05-03T17:52:32.690506+0000","last_unstale":"2024-05-03T17:53:19.872477+0000","last_undegraded":"2024-05-03T17:53:19.872477+0000","last_fullsized":"2024-05-03T17:53:19.872477+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:19.810205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869511+0000","last_change":"2024-05-03T17:52:35.718229+0000","last_active":"2024-05-03T17:53:19.869511+0000","last_peered":"2024-05-03T17:53:19.869511+0000","last_clean":"2024-05-03T17:53:19.869511+0000","last_became_active":"2024-05-03T17:52:35.717907+0000","last_became_peered":"2024-05-03T17:52:35.717907+0000","last_unstale":"2024-05-03T17:53:19.869511+0000","last_undegraded":"2024-05-03T17:53:19.869511+0000","last_fullsized":"2024-05-03T17:53:19.869511+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:45.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":73,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:35.826473+0000","last_change":"2024-05-03T17:52:32.691174+0000","last_active":"2024-05-03T17:53:35.826473+0000","last_peered":"2024-05-03T17:53:35.826473+0000","last_clean":"2024-05-03T17:53:35.826473+0000","last_became_active":"2024-05-03T17:52:32.689907+0000","last_became_peered":"2024-05-03T17:52:32.689907+0000","last_unstale":"2024-05-03T17:53:35.826473+0000","last_undegraded":"2024-05-03T17:53:35.826473+0000","last_fullsized":"2024-05-03T17:53:35.826473+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:46.630436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867946+0000","last_change":"2024-05-03T17:52:35.720290+0000","last_active":"2024-05-03T17:53:19.867946+0000","last_peered":"2024-05-03T17:53:19.867946+0000","last_clean":"2024-05-03T17:53:19.867946+0000","last_became_active":"2024-05-03T17:52:35.719962+0000","last_became_peered":"2024-05-03T17:52:35.719962+0000","last_unstale":"2024-05-03T17:53:19.867946+0000","last_undegraded":"2024-05-03T17:53:19.867946+0000","last_fullsized":"2024-05-03T17:53:19.867946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:31:41.301592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077710+0000","last_change":"2024-05-03T17:52:32.689937+0000","last_active":"2024-05-03T17:53:20.077710+0000","last_peered":"2024-05-03T17:53:20.077710+0000","last_clean":"2024-05-03T17:53:20.077710+0000","last_became_active":"2024-05-03T17:52:32.689624+0000","last_became_peered":"2024-05-03T17:52:32.689624+0000","last_unstale":"2024-05-03T17:53:20.077710+0000","last_undegraded":"2024-05-03T17:53:20.077710+0000","last_fullsized":"2024-05-03T17:53:20.077710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:24.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870881+0000","last_change":"2024-05-03T17:52:35.726824+0000","last_active":"2024-05-03T17:53:19.870881+0000","last_peered":"2024-05-03T17:53:19.870881+0000","last_clean":"2024-05-03T17:53:19.870881+0000","last_became_active":"2024-05-03T17:52:35.726494+0000","last_became_peered":"2024-05-03T17:52:35.726494+0000","last_unstale":"2024-05-03T17:53:19.870881+0000","last_undegraded":"2024-05-03T17:53:19.870881+0000","last_fullsized":"2024-05-03T17:53:19.870881+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:41:21.284852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873051+0000","last_change":"2024-05-03T17:52:32.935471+0000","last_active":"2024-05-03T17:53:19.873051+0000","last_peered":"2024-05-03T17:53:19.873051+0000","last_clean":"2024-05-03T17:53:19.873051+0000","last_became_active":"2024-05-03T17:52:32.935012+0000","last_became_peered":"2024-05-03T17:52:32.935012+0000","last_unstale":"2024-05-03T17:53:19.873051+0000","last_undegraded":"2024-05-03T17:53:19.873051+0000","last_fullsized":"2024-05-03T17:53:19.873051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:35:38.654577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867634+0000","last_change":"2024-05-03T17:52:35.727124+0000","last_active":"2024-05-03T17:53:19.867634+0000","last_peered":"2024-05-03T17:53:19.867634+0000","last_clean":"2024-05-03T17:53:19.867634+0000","last_became_active":"2024-05-03T17:52:35.726804+0000","last_became_peered":"2024-05-03T17:52:35.726804+0000","last_unstale":"2024-05-03T17:53:19.867634+0000","last_undegraded":"2024-05-03T17:53:19.867634+0000","last_fullsized":"2024-05-03T17:53:19.867634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:32.000803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'29","reported_seq":152,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:36.827616+0000","last_change":"2024-05-03T17:52:32.685014+0000","last_active":"2024-05-03T17:53:36.827616+0000","last_peered":"2024-05-03T17:53:36.827616+0000","last_clean":"2024-05-03T17:53:36.827616+0000","last_became_active":"2024-05-03T17:52:32.684660+0000","last_became_peered":"2024-05-03T17:52:32.684660+0000","last_unstale":"2024-05-03T17:53:36.827616+0000","last_undegraded":"2024-05-03T17:53:36.827616+0000","last_fullsized":"2024-05-03T17:53:36.827616+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:25:56.890283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54817,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":97,"num_read_kb":101,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871281+0000","last_change":"2024-05-03T17:52:35.733675+0000","last_active":"2024-05-03T17:53:19.871281+0000","last_peered":"2024-05-03T17:53:19.871281+0000","last_clean":"2024-05-03T17:53:19.871281+0000","last_became_active":"2024-05-03T17:52:35.733234+0000","last_became_peered":"2024-05-03T17:52:35.733234+0000","last_unstale":"2024-05-03T17:53:19.871281+0000","last_undegraded":"2024-05-03T17:53:19.871281+0000","last_fullsized":"2024-05-03T17:53:19.871281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:57.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886182+0000","last_change":"2024-05-03T17:52:32.690462+0000","last_active":"2024-05-03T17:53:19.886182+0000","last_peered":"2024-05-03T17:53:19.886182+0000","last_clean":"2024-05-03T17:53:19.886182+0000","last_became_active":"2024-05-03T17:52:32.690169+0000","last_became_peered":"2024-05-03T17:52:32.690169+0000","last_unstale":"2024-05-03T17:53:19.886182+0000","last_undegraded":"2024-05-03T17:53:19.886182+0000","last_fullsized":"2024-05-03T17:53:19.886182+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:59.324612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867010+0000","last_change":"2024-05-03T17:52:35.720730+0000","last_active":"2024-05-03T17:53:19.867010+0000","last_peered":"2024-05-03T17:53:19.867010+0000","last_clean":"2024-05-03T17:53:19.867010+0000","last_became_active":"2024-05-03T17:52:35.720392+0000","last_became_peered":"2024-05-03T17:52:35.720392+0000","last_unstale":"2024-05-03T17:53:19.867010+0000","last_undegraded":"2024-05-03T17:53:19.867010+0000","last_fullsized":"2024-05-03T17:53:19.867010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:58.528148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872742+0000","last_change":"2024-05-03T17:52:32.691439+0000","last_active":"2024-05-03T17:53:19.872742+0000","last_peered":"2024-05-03T17:53:19.872742+0000","last_clean":"2024-05-03T17:53:19.872742+0000","last_became_active":"2024-05-03T17:52:32.691080+0000","last_became_peered":"2024-05-03T17:52:32.691080+0000","last_unstale":"2024-05-03T17:53:19.872742+0000","last_undegraded":"2024-05-03T17:53:19.872742+0000","last_fullsized":"2024-05-03T17:53:19.872742+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:40:43.120194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077160+0000","last_change":"2024-05-03T17:52:35.729860+0000","last_active":"2024-05-03T17:53:20.077160+0000","last_peered":"2024-05-03T17:53:20.077160+0000","last_clean":"2024-05-03T17:53:20.077160+0000","last_became_active":"2024-05-03T17:52:35.729508+0000","last_became_peered":"2024-05-03T17:52:35.729508+0000","last_unstale":"2024-05-03T17:53:20.077160+0000","last_undegraded":"2024-05-03T17:53:20.077160+0000","last_fullsized":"2024-05-03T17:53:20.077160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:39:20.859215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":74,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:32.822388+0000","last_change":"2024-05-03T17:52:32.689346+0000","last_active":"2024-05-03T17:53:32.822388+0000","last_peered":"2024-05-03T17:53:32.822388+0000","last_clean":"2024-05-03T17:53:32.822388+0000","last_became_active":"2024-05-03T17:52:32.689009+0000","last_became_peered":"2024-05-03T17:52:32.689009+0000","last_unstale":"2024-05-03T17:53:32.822388+0000","last_undegraded":"2024-05-03T17:53:32.822388+0000","last_fullsized":"2024-05-03T17:53:32.822388+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:55:36.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077075+0000","last_change":"2024-05-03T17:52:35.729226+0000","last_active":"2024-05-03T17:53:20.077075+0000","last_peered":"2024-05-03T17:53:20.077075+0000","last_clean":"2024-05-03T17:53:20.077075+0000","last_became_active":"2024-05-03T17:52:35.728846+0000","last_became_peered":"2024-05-03T17:52:35.728846+0000","last_unstale":"2024-05-03T17:53:20.077075+0000","last_undegraded":"2024-05-03T17:53:20.077075+0000","last_fullsized":"2024-05-03T17:53:20.077075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:39:16.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870410+0000","last_change":"2024-05-03T17:52:32.936108+0000","last_active":"2024-05-03T17:53:19.870410+0000","last_peered":"2024-05-03T17:53:19.870410+0000","last_clean":"2024-05-03T17:53:19.870410+0000","last_became_active":"2024-05-03T17:52:32.935782+0000","last_became_peered":"2024-05-03T17:52:32.935782+0000","last_unstale":"2024-05-03T17:53:19.870410+0000","last_undegraded":"2024-05-03T17:53:19.870410+0000","last_fullsized":"2024-05-03T17:53:19.870410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:47.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872695+0000","last_change":"2024-05-03T17:52:35.727163+0000","last_active":"2024-05-03T17:53:19.872695+0000","last_peered":"2024-05-03T17:53:19.872695+0000","last_clean":"2024-05-03T17:53:19.872695+0000","last_became_active":"2024-05-03T17:52:35.726824+0000","last_became_peered":"2024-05-03T17:52:35.726824+0000","last_unstale":"2024-05-03T17:53:19.872695+0000","last_undegraded":"2024-05-03T17:53:19.872695+0000","last_fullsized":"2024-05-03T17:53:19.872695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:24:50.803860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871546+0000","last_change":"2024-05-03T17:52:32.687329+0000","last_active":"2024-05-03T17:53:19.871546+0000","last_peered":"2024-05-03T17:53:19.871546+0000","last_clean":"2024-05-03T17:53:19.871546+0000","last_became_active":"2024-05-03T17:52:32.687025+0000","last_became_peered":"2024-05-03T17:52:32.687025+0000","last_unstale":"2024-05-03T17:53:19.871546+0000","last_undegraded":"2024-05-03T17:53:19.871546+0000","last_fullsized":"2024-05-03T17:53:19.871546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:42:15.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869955+0000","last_change":"2024-05-03T17:52:35.728797+0000","last_active":"2024-05-03T17:53:19.869955+0000","last_peered":"2024-05-03T17:53:19.869955+0000","last_clean":"2024-05-03T17:53:19.869955+0000","last_became_active":"2024-05-03T17:52:35.728429+0000","last_became_peered":"2024-05-03T17:52:35.728429+0000","last_unstale":"2024-05-03T17:53:19.869955+0000","last_undegraded":"2024-05-03T17:53:19.869955+0000","last_fullsized":"2024-05-03T17:53:19.869955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:19.584593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872508+0000","last_change":"2024-05-03T17:53:19.872508+0000","last_active":"2024-05-03T17:53:19.872508+0000","last_peered":"2024-05-03T17:53:19.872508+0000","last_clean":"2024-05-03T17:53:19.872508+0000","last_became_active":"2024-05-03T17:53:19.872175+0000","last_became_peered":"2024-05-03T17:53:19.872175+0000","last_unstale":"2024-05-03T17:53:19.872508+0000","last_undegraded":"2024-05-03T17:53:19.872508+0000","last_fullsized":"2024-05-03T17:53:19.872508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:24:08.246884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871588+0000","last_change":"2024-05-03T17:52:35.719115+0000","last_active":"2024-05-03T17:53:19.871588+0000","last_peered":"2024-05-03T17:53:19.871588+0000","last_clean":"2024-05-03T17:53:19.871588+0000","last_became_active":"2024-05-03T17:52:35.718054+0000","last_became_peered":"2024-05-03T17:52:35.718054+0000","last_unstale":"2024-05-03T17:53:19.871588+0000","last_undegraded":"2024-05-03T17:53:19.871588+0000","last_fullsized":"2024-05-03T17:53:19.871588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.903766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077572+0000","last_change":"2024-05-03T17:52:32.688993+0000","last_active":"2024-05-03T17:53:20.077572+0000","last_peered":"2024-05-03T17:53:20.077572+0000","last_clean":"2024-05-03T17:53:20.077572+0000","last_became_active":"2024-05-03T17:52:32.688706+0000","last_became_peered":"2024-05-03T17:52:32.688706+0000","last_unstale":"2024-05-03T17:53:20.077572+0000","last_undegraded":"2024-05-03T17:53:20.077572+0000","last_fullsized":"2024-05-03T17:53:20.077572+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:40:14.023338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872957+0000","last_change":"2024-05-03T17:53:19.872957+0000","last_active":"2024-05-03T17:53:19.872957+0000","last_peered":"2024-05-03T17:53:19.872957+0000","last_clean":"2024-05-03T17:53:19.872957+0000","last_became_active":"2024-05-03T17:53:19.872690+0000","last_became_peered":"2024-05-03T17:53:19.872690+0000","last_unstale":"2024-05-03T17:53:19.872957+0000","last_undegraded":"2024-05-03T17:53:19.872957+0000","last_fullsized":"2024-05-03T17:53:19.872957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:53:43.801983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872998+0000","last_change":"2024-05-03T17:52:35.728958+0000","last_active":"2024-05-03T17:53:19.872998+0000","last_peered":"2024-05-03T17:53:19.872998+0000","last_clean":"2024-05-03T17:53:19.872998+0000","last_became_active":"2024-05-03T17:52:35.728619+0000","last_became_peered":"2024-05-03T17:52:35.728619+0000","last_unstale":"2024-05-03T17:53:19.872998+0000","last_undegraded":"2024-05-03T17:53:19.872998+0000","last_fullsized":"2024-05-03T17:53:19.872998+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:29:29.922324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":75,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:36.827523+0000","last_change":"2024-05-03T17:52:32.935367+0000","last_active":"2024-05-03T17:53:36.827523+0000","last_peered":"2024-05-03T17:53:36.827523+0000","last_clean":"2024-05-03T17:53:36.827523+0000","last_became_active":"2024-05-03T17:52:32.934938+0000","last_became_peered":"2024-05-03T17:52:32.934938+0000","last_unstale":"2024-05-03T17:53:36.827523+0000","last_undegraded":"2024-05-03T17:53:36.827523+0000","last_fullsized":"2024-05-03T17:53:36.827523+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:07:21.642883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076663+0000","last_change":"2024-05-03T17:52:35.731035+0000","last_active":"2024-05-03T17:53:20.076663+0000","last_peered":"2024-05-03T17:53:20.076663+0000","last_clean":"2024-05-03T17:53:20.076663+0000","last_became_active":"2024-05-03T17:52:35.730034+0000","last_became_peered":"2024-05-03T17:52:35.730034+0000","last_unstale":"2024-05-03T17:53:20.076663+0000","last_undegraded":"2024-05-03T17:53:20.076663+0000","last_fullsized":"2024-05-03T17:53:20.076663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:25.545290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077849+0000","last_change":"2024-05-03T17:52:32.690043+0000","last_active":"2024-05-03T17:53:20.077849+0000","last_peered":"2024-05-03T17:53:20.077849+0000","last_clean":"2024-05-03T17:53:20.077849+0000","last_became_active":"2024-05-03T17:52:32.689562+0000","last_became_peered":"2024-05-03T17:52:32.689562+0000","last_unstale":"2024-05-03T17:53:20.077849+0000","last_undegraded":"2024-05-03T17:53:20.077849+0000","last_fullsized":"2024-05-03T17:53:20.077849+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:59.014968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":97,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077303+0000","last_change":"2024-05-03T17:51:02.340854+0000","last_active":"2024-05-03T17:53:20.077303+0000","last_peered":"2024-05-03T17:53:20.077303+0000","last_clean":"2024-05-03T17:53:20.077303+0000","last_became_active":"2024-05-03T17:51:02.336454+0000","last_became_peered":"2024-05-03T17:51:02.336454+0000","last_unstale":"2024-05-03T17:53:20.077303+0000","last_undegraded":"2024-05-03T17:53:20.077303+0000","last_fullsized":"2024-05-03T17:53:20.077303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_clean_scrub_stamp":"2024-05-03T17:47:41.306771+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:18:08.512098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.887126+0000","last_change":"2024-05-03T17:53:19.887126+0000","last_active":"2024-05-03T17:53:19.887126+0000","last_peered":"2024-05-03T17:53:19.887126+0000","last_clean":"2024-05-03T17:53:19.887126+0000","last_became_active":"2024-05-03T17:53:19.886736+0000","last_became_peered":"2024-05-03T17:53:19.886736+0000","last_unstale":"2024-05-03T17:53:19.887126+0000","last_undegraded":"2024-05-03T17:53:19.887126+0000","last_fullsized":"2024-05-03T17:53:19.887126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:35:02.036825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077402+0000","last_change":"2024-05-03T17:52:32.690093+0000","last_active":"2024-05-03T17:53:20.077402+0000","last_peered":"2024-05-03T17:53:20.077402+0000","last_clean":"2024-05-03T17:53:20.077402+0000","last_became_active":"2024-05-03T17:52:32.689777+0000","last_became_peered":"2024-05-03T17:52:32.689777+0000","last_unstale":"2024-05-03T17:53:20.077402+0000","last_undegraded":"2024-05-03T17:53:20.077402+0000","last_fullsized":"2024-05-03T17:53:20.077402+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:09.693011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871341+0000","last_change":"2024-05-03T17:52:32.935589+0000","last_active":"2024-05-03T17:53:19.871341+0000","last_peered":"2024-05-03T17:53:19.871341+0000","last_clean":"2024-05-03T17:53:19.871341+0000","last_became_active":"2024-05-03T17:52:32.935221+0000","last_became_peered":"2024-05-03T17:52:32.935221+0000","last_unstale":"2024-05-03T17:53:19.871341+0000","last_undegraded":"2024-05-03T17:53:19.871341+0000","last_fullsized":"2024-05-03T17:53:19.871341+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:26:08.001761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867191+0000","last_change":"2024-05-03T17:52:35.724293+0000","last_active":"2024-05-03T17:53:19.867191+0000","last_peered":"2024-05-03T17:53:19.867191+0000","last_clean":"2024-05-03T17:53:19.867191+0000","last_became_active":"2024-05-03T17:52:35.723978+0000","last_became_peered":"2024-05-03T17:52:35.723978+0000","last_unstale":"2024-05-03T17:53:19.867191+0000","last_undegraded":"2024-05-03T17:53:19.867191+0000","last_fullsized":"2024-05-03T17:53:19.867191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:23.357872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869626+0000","last_change":"2024-05-03T17:52:32.687303+0000","last_active":"2024-05-03T17:53:19.869626+0000","last_peered":"2024-05-03T17:53:19.869626+0000","last_clean":"2024-05-03T17:53:19.869626+0000","last_became_active":"2024-05-03T17:52:32.686827+0000","last_became_peered":"2024-05-03T17:52:32.686827+0000","last_unstale":"2024-05-03T17:53:19.869626+0000","last_undegraded":"2024-05-03T17:53:19.869626+0000","last_fullsized":"2024-05-03T17:53:19.869626+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:11:53.565215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076856+0000","last_change":"2024-05-03T17:52:35.720898+0000","last_active":"2024-05-03T17:53:20.076856+0000","last_peered":"2024-05-03T17:53:20.076856+0000","last_clean":"2024-05-03T17:53:20.076856+0000","last_became_active":"2024-05-03T17:52:35.720572+0000","last_became_peered":"2024-05-03T17:52:35.720572+0000","last_unstale":"2024-05-03T17:53:20.076856+0000","last_undegraded":"2024-05-03T17:53:20.076856+0000","last_fullsized":"2024-05-03T17:53:20.076856+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:01.697167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077357+0000","last_change":"2024-05-03T17:52:33.249970+0000","last_active":"2024-05-03T17:53:20.077357+0000","last_peered":"2024-05-03T17:53:20.077357+0000","last_clean":"2024-05-03T17:53:20.077357+0000","last_became_active":"2024-05-03T17:52:33.249596+0000","last_became_peered":"2024-05-03T17:52:33.249596+0000","last_unstale":"2024-05-03T17:53:20.077357+0000","last_undegraded":"2024-05-03T17:53:20.077357+0000","last_fullsized":"2024-05-03T17:53:20.077357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:46:53.058586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870807+0000","last_change":"2024-05-03T17:52:35.722129+0000","last_active":"2024-05-03T17:53:19.870807+0000","last_peered":"2024-05-03T17:53:19.870807+0000","last_clean":"2024-05-03T17:53:19.870807+0000","last_became_active":"2024-05-03T17:52:35.721815+0000","last_became_peered":"2024-05-03T17:52:35.721815+0000","last_unstale":"2024-05-03T17:53:19.870807+0000","last_undegraded":"2024-05-03T17:53:19.870807+0000","last_fullsized":"2024-05-03T17:53:19.870807+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:33.781350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867376+0000","last_change":"2024-05-03T17:52:32.936157+0000","last_active":"2024-05-03T17:53:19.867376+0000","last_peered":"2024-05-03T17:53:19.867376+0000","last_clean":"2024-05-03T17:53:19.867376+0000","last_became_active":"2024-05-03T17:52:32.935842+0000","last_became_peered":"2024-05-03T17:52:32.935842+0000","last_unstale":"2024-05-03T17:53:19.867376+0000","last_undegraded":"2024-05-03T17:53:19.867376+0000","last_fullsized":"2024-05-03T17:53:19.867376+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:18.006451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870494+0000","last_change":"2024-05-03T17:52:35.735993+0000","last_active":"2024-05-03T17:53:19.870494+0000","last_peered":"2024-05-03T17:53:19.870494+0000","last_clean":"2024-05-03T17:53:19.870494+0000","last_became_active":"2024-05-03T17:52:35.735612+0000","last_became_peered":"2024-05-03T17:52:35.735612+0000","last_unstale":"2024-05-03T17:53:19.870494+0000","last_undegraded":"2024-05-03T17:53:19.870494+0000","last_fullsized":"2024-05-03T17:53:19.870494+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:59.216022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076573+0000","last_change":"2024-05-03T17:52:32.687147+0000","last_active":"2024-05-03T17:53:20.076573+0000","last_peered":"2024-05-03T17:53:20.076573+0000","last_clean":"2024-05-03T17:53:20.076573+0000","last_became_active":"2024-05-03T17:52:32.686831+0000","last_became_peered":"2024-05-03T17:52:32.686831+0000","last_unstale":"2024-05-03T17:53:20.076573+0000","last_undegraded":"2024-05-03T17:53:20.076573+0000","last_fullsized":"2024-05-03T17:53:20.076573+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:24:30.092829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869678+0000","last_change":"2024-05-03T17:52:35.721862+0000","last_active":"2024-05-03T17:53:19.869678+0000","last_peered":"2024-05-03T17:53:19.869678+0000","last_clean":"2024-05-03T17:53:19.869678+0000","last_became_active":"2024-05-03T17:52:35.721393+0000","last_became_peered":"2024-05-03T17:52:35.721393+0000","last_unstale":"2024-05-03T17:53:19.869678+0000","last_undegraded":"2024-05-03T17:53:19.869678+0000","last_fullsized":"2024-05-03T17:53:19.869678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:47:32.014817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885948+0000","last_change":"2024-05-03T17:52:32.687867+0000","last_active":"2024-05-03T17:53:19.885948+0000","last_peered":"2024-05-03T17:53:19.885948+0000","last_clean":"2024-05-03T17:53:19.885948+0000","last_became_active":"2024-05-03T17:52:32.687557+0000","last_became_peered":"2024-05-03T17:52:32.687557+0000","last_unstale":"2024-05-03T17:53:19.885948+0000","last_undegraded":"2024-05-03T17:53:19.885948+0000","last_fullsized":"2024-05-03T17:53:19.885948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:45.863180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076840+0000","last_change":"2024-05-03T17:52:35.731057+0000","last_active":"2024-05-03T17:53:20.076840+0000","last_peered":"2024-05-03T17:53:20.076840+0000","last_clean":"2024-05-03T17:53:20.076840+0000","last_became_active":"2024-05-03T17:52:35.729293+0000","last_became_peered":"2024-05-03T17:52:35.729293+0000","last_unstale":"2024-05-03T17:53:20.076840+0000","last_undegraded":"2024-05-03T17:53:20.076840+0000","last_fullsized":"2024-05-03T17:53:20.076840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:12:53.888078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870721+0000","last_change":"2024-05-03T17:52:32.689675+0000","last_active":"2024-05-03T17:53:19.870721+0000","last_peered":"2024-05-03T17:53:19.870721+0000","last_clean":"2024-05-03T17:53:19.870721+0000","last_became_active":"2024-05-03T17:52:32.689340+0000","last_became_peered":"2024-05-03T17:52:32.689340+0000","last_unstale":"2024-05-03T17:53:19.870721+0000","last_undegraded":"2024-05-03T17:53:19.870721+0000","last_fullsized":"2024-05-03T17:53:19.870721+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:31.398045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077600+0000","last_change":"2024-05-03T17:52:35.728228+0000","last_active":"2024-05-03T17:53:20.077600+0000","last_peered":"2024-05-03T17:53:20.077600+0000","last_clean":"2024-05-03T17:53:20.077600+0000","last_became_active":"2024-05-03T17:52:35.727233+0000","last_became_peered":"2024-05-03T17:52:35.727233+0000","last_unstale":"2024-05-03T17:53:20.077600+0000","last_undegraded":"2024-05-03T17:53:20.077600+0000","last_fullsized":"2024-05-03T17:53:20.077600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:57.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886416+0000","last_change":"2024-05-03T17:52:33.249690+0000","last_active":"2024-05-03T17:53:19.886416+0000","last_peered":"2024-05-03T17:53:19.886416+0000","last_clean":"2024-05-03T17:53:19.886416+0000","last_became_active":"2024-05-03T17:52:33.249302+0000","last_became_peered":"2024-05-03T17:52:33.249302+0000","last_unstale":"2024-05-03T17:53:19.886416+0000","last_undegraded":"2024-05-03T17:53:19.886416+0000","last_fullsized":"2024-05-03T17:53:19.886416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:05:48.700670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077737+0000","last_change":"2024-05-03T17:52:35.727171+0000","last_active":"2024-05-03T17:53:20.077737+0000","last_peered":"2024-05-03T17:53:20.077737+0000","last_clean":"2024-05-03T17:53:20.077737+0000","last_became_active":"2024-05-03T17:52:35.726834+0000","last_became_peered":"2024-05-03T17:52:35.726834+0000","last_unstale":"2024-05-03T17:53:20.077737+0000","last_undegraded":"2024-05-03T17:53:20.077737+0000","last_fullsized":"2024-05-03T17:53:20.077737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:40:09.938183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872806+0000","last_change":"2024-05-03T17:52:32.693694+0000","last_active":"2024-05-03T17:53:19.872806+0000","last_peered":"2024-05-03T17:53:19.872806+0000","last_clean":"2024-05-03T17:53:19.872806+0000","last_became_active":"2024-05-03T17:52:32.693310+0000","last_became_peered":"2024-05-03T17:52:32.693310+0000","last_unstale":"2024-05-03T17:53:19.872806+0000","last_undegraded":"2024-05-03T17:53:19.872806+0000","last_fullsized":"2024-05-03T17:53:19.872806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:59:37.449434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077538+0000","last_change":"2024-05-03T17:52:35.728267+0000","last_active":"2024-05-03T17:53:20.077538+0000","last_peered":"2024-05-03T17:53:20.077538+0000","last_clean":"2024-05-03T17:53:20.077538+0000","last_became_active":"2024-05-03T17:52:35.727902+0000","last_became_peered":"2024-05-03T17:52:35.727902+0000","last_unstale":"2024-05-03T17:53:20.077538+0000","last_undegraded":"2024-05-03T17:53:20.077538+0000","last_fullsized":"2024-05-03T17:53:20.077538+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:09.223850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867146+0000","last_change":"2024-05-03T17:52:32.687248+0000","last_active":"2024-05-03T17:53:19.867146+0000","last_peered":"2024-05-03T17:53:19.867146+0000","last_clean":"2024-05-03T17:53:19.867146+0000","last_became_active":"2024-05-03T17:52:32.686921+0000","last_became_peered":"2024-05-03T17:52:32.686921+0000","last_unstale":"2024-05-03T17:53:19.867146+0000","last_undegraded":"2024-05-03T17:53:19.867146+0000","last_fullsized":"2024-05-03T17:53:19.867146+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:35:40.849484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077858+0000","last_change":"2024-05-03T17:52:35.729383+0000","last_active":"2024-05-03T17:53:20.077858+0000","last_peered":"2024-05-03T17:53:20.077858+0000","last_clean":"2024-05-03T17:53:20.077858+0000","last_became_active":"2024-05-03T17:52:35.729037+0000","last_became_peered":"2024-05-03T17:52:35.729037+0000","last_unstale":"2024-05-03T17:53:20.077858+0000","last_undegraded":"2024-05-03T17:53:20.077858+0000","last_fullsized":"2024-05-03T17:53:20.077858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:03:03.815770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869741+0000","last_change":"2024-05-03T17:52:35.731809+0000","last_active":"2024-05-03T17:53:19.869741+0000","last_peered":"2024-05-03T17:53:19.869741+0000","last_clean":"2024-05-03T17:53:19.869741+0000","last_became_active":"2024-05-03T17:52:35.729360+0000","last_became_peered":"2024-05-03T17:52:35.729360+0000","last_unstale":"2024-05-03T17:53:19.869741+0000","last_undegraded":"2024-05-03T17:53:19.869741+0000","last_fullsized":"2024-05-03T17:53:19.869741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:22.181184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886168+0000","last_change":"2024-05-03T17:52:35.727771+0000","last_active":"2024-05-03T17:53:19.886168+0000","last_peered":"2024-05-03T17:53:19.886168+0000","last_clean":"2024-05-03T17:53:19.886168+0000","last_became_active":"2024-05-03T17:52:35.727414+0000","last_became_peered":"2024-05-03T17:52:35.727414+0000","last_unstale":"2024-05-03T17:53:19.886168+0000","last_undegraded":"2024-05-03T17:53:19.886168+0000","last_fullsized":"2024-05-03T17:53:19.886168+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:09:02.905481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885867+0000","last_change":"2024-05-03T17:52:35.727433+0000","last_active":"2024-05-03T17:53:19.885867+0000","last_peered":"2024-05-03T17:53:19.885867+0000","last_clean":"2024-05-03T17:53:19.885867+0000","last_became_active":"2024-05-03T17:52:35.727123+0000","last_became_peered":"2024-05-03T17:52:35.727123+0000","last_unstale":"2024-05-03T17:53:19.885867+0000","last_undegraded":"2024-05-03T17:53:19.885867+0000","last_fullsized":"2024-05-03T17:53:19.885867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:41:36.871788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870666+0000","last_change":"2024-05-03T17:52:35.721332+0000","last_active":"2024-05-03T17:53:19.870666+0000","last_peered":"2024-05-03T17:53:19.870666+0000","last_clean":"2024-05-03T17:53:19.870666+0000","last_became_active":"2024-05-03T17:52:35.721004+0000","last_became_peered":"2024-05-03T17:52:35.721004+0000","last_unstale":"2024-05-03T17:53:19.870666+0000","last_undegraded":"2024-05-03T17:53:19.870666+0000","last_fullsized":"2024-05-03T17:53:19.870666+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:35:03.443163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867305+0000","last_change":"2024-05-03T17:52:35.723649+0000","last_active":"2024-05-03T17:53:19.867305+0000","last_peered":"2024-05-03T17:53:19.867305+0000","last_clean":"2024-05-03T17:53:19.867305+0000","last_became_active":"2024-05-03T17:52:35.723303+0000","last_became_peered":"2024-05-03T17:52:35.723303+0000","last_unstale":"2024-05-03T17:53:19.867305+0000","last_undegraded":"2024-05-03T17:53:19.867305+0000","last_fullsized":"2024-05-03T17:53:19.867305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:02:49.093039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870096+0000","last_change":"2024-05-03T17:52:35.732505+0000","last_active":"2024-05-03T17:53:19.870096+0000","last_peered":"2024-05-03T17:53:19.870096+0000","last_clean":"2024-05-03T17:53:19.870096+0000","last_became_active":"2024-05-03T17:52:35.732180+0000","last_became_peered":"2024-05-03T17:52:35.732180+0000","last_unstale":"2024-05-03T17:53:19.870096+0000","last_undegraded":"2024-05-03T17:53:19.870096+0000","last_fullsized":"2024-05-03T17:53:19.870096+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.371260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886349+0000","last_change":"2024-05-03T17:52:35.732217+0000","last_active":"2024-05-03T17:53:19.886349+0000","last_peered":"2024-05-03T17:53:19.886349+0000","last_clean":"2024-05-03T17:53:19.886349+0000","last_became_active":"2024-05-03T17:52:35.731868+0000","last_became_peered":"2024-05-03T17:52:35.731868+0000","last_unstale":"2024-05-03T17:53:19.886349+0000","last_undegraded":"2024-05-03T17:53:19.886349+0000","last_fullsized":"2024-05-03T17:53:19.886349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:04:17.521528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076813+0000","last_change":"2024-05-03T17:52:35.723641+0000","last_active":"2024-05-03T17:53:20.076813+0000","last_peered":"2024-05-03T17:53:20.076813+0000","last_clean":"2024-05-03T17:53:20.076813+0000","last_became_active":"2024-05-03T17:52:35.723323+0000","last_became_peered":"2024-05-03T17:52:35.723323+0000","last_unstale":"2024-05-03T17:53:20.076813+0000","last_undegraded":"2024-05-03T17:53:20.076813+0000","last_fullsized":"2024-05-03T17:53:20.076813+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:45:34.032407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872477+0000","last_change":"2024-05-03T17:52:35.731008+0000","last_active":"2024-05-03T17:53:19.872477+0000","last_peered":"2024-05-03T17:53:19.872477+0000","last_clean":"2024-05-03T17:53:19.872477+0000","last_became_active":"2024-05-03T17:52:35.730627+0000","last_became_peered":"2024-05-03T17:52:35.730627+0000","last_unstale":"2024-05-03T17:53:19.872477+0000","last_undegraded":"2024-05-03T17:53:19.872477+0000","last_fullsized":"2024-05-03T17:53:19.872477+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:30:32.884518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076519+0000","last_change":"2024-05-03T17:52:35.726938+0000","last_active":"2024-05-03T17:53:20.076519+0000","last_peered":"2024-05-03T17:53:20.076519+0000","last_clean":"2024-05-03T17:53:20.076519+0000","last_became_active":"2024-05-03T17:52:35.726611+0000","last_became_peered":"2024-05-03T17:52:35.726611+0000","last_unstale":"2024-05-03T17:53:20.076519+0000","last_undegraded":"2024-05-03T17:53:20.076519+0000","last_fullsized":"2024-05-03T17:53:20.076519+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:11.587035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867632+0000","last_change":"2024-05-03T17:52:35.722562+0000","last_active":"2024-05-03T17:53:19.867632+0000","last_peered":"2024-05-03T17:53:19.867632+0000","last_clean":"2024-05-03T17:53:19.867632+0000","last_became_active":"2024-05-03T17:52:35.722229+0000","last_became_peered":"2024-05-03T17:52:35.722229+0000","last_unstale":"2024-05-03T17:53:19.867632+0000","last_undegraded":"2024-05-03T17:53:19.867632+0000","last_fullsized":"2024-05-03T17:53:19.867632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:57:06.730533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867188+0000","last_change":"2024-05-03T17:52:35.722707+0000","last_active":"2024-05-03T17:53:19.867188+0000","last_peered":"2024-05-03T17:53:19.867188+0000","last_clean":"2024-05-03T17:53:19.867188+0000","last_became_active":"2024-05-03T17:52:35.721847+0000","last_became_peered":"2024-05-03T17:52:35.721847+0000","last_unstale":"2024-05-03T17:53:19.867188+0000","last_undegraded":"2024-05-03T17:53:19.867188+0000","last_fullsized":"2024-05-03T17:53:19.867188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:56:21.620074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870547+0000","last_change":"2024-05-03T17:52:35.731810+0000","last_active":"2024-05-03T17:53:19.870547+0000","last_peered":"2024-05-03T17:53:19.870547+0000","last_clean":"2024-05-03T17:53:19.870547+0000","last_became_active":"2024-05-03T17:52:35.728986+0000","last_became_peered":"2024-05-03T17:52:35.728986+0000","last_unstale":"2024-05-03T17:53:19.870547+0000","last_undegraded":"2024-05-03T17:53:19.870547+0000","last_fullsized":"2024-05-03T17:53:19.870547+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:18.275792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280811,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59209,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":288,"num_read_kb":210,"num_write":161,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":177627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743452,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":417972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906978,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1064661,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070503,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":418208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266731,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":418128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462961,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":1010449,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626486,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":417597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":913408,"data_stored":472207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":418868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":473948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":872448,"data_stored":417908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":860160,"data_stored":418425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822715,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":1009742,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T17:53:39.452 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-03T17:53:39.453 INFO:tasks.cephadm:Setting up client nodes... 2024-05-03T17:53:39.453 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-03T17:53:39.701 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:40.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:40 smithi122 ceph-mon[29744]: pgmap v319: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T17:53:40.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:40 smithi122 ceph-mon[29744]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:40.536 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:40 smithi149 ceph-mon[33409]: pgmap v319: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T17:53:40.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:40 smithi149 ceph-mon[33409]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:40.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:40 smithi192 ceph-mon[33753]: pgmap v319: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T17:53:40.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:40 smithi192 ceph-mon[33753]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:53:41.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.433+0000 7ffac50a7700 1 -- 172.21.15.122:0/383807807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac0106940 msgr2=0x7ffac0108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:41.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.433+0000 7ffac50a7700 1 --2- 172.21.15.122:0/383807807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac0106940 0x7ffac0108d30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009990 tx=0x7ffab4009ca0 comp rx=0 tx=0).stop 2024-05-03T17:53:41.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.433+0000 7ffac50a7700 1 -- 172.21.15.122:0/383807807 shutdown_connections 2024-05-03T17:53:41.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.433+0000 7ffac50a7700 1 --2- 172.21.15.122:0/383807807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffac0109270 0x7ffac010b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:41.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.433+0000 7ffac50a7700 1 --2- 172.21.15.122:0/383807807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac0104010 0x7ffac0106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:41.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.433+0000 7ffac50a7700 1 --2- 172.21.15.122:0/383807807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac0106940 0x7ffac0108d30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:41.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.433+0000 7ffac50a7700 1 -- 172.21.15.122:0/383807807 >> 172.21.15.122:0/383807807 conn(0x7ffac00fd9c0 msgr2=0x7ffac00ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.433+0000 7ffac50a7700 1 -- 172.21.15.122:0/383807807 shutdown_connections 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.433+0000 7ffac50a7700 1 -- 172.21.15.122:0/383807807 wait complete. 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffac50a7700 1 Processor -- start 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffac50a7700 1 -- start start 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffac50a7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffac0104010 0x7ffac0199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffac50a7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac0106940 0x7ffac0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffac50a7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac0109270 0x7ffac01a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffac50a7700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffac010efa0 con 0x7ffac0109270 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffac50a7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffac010ee20 con 0x7ffac0106940 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffac50a7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffac010f120 con 0x7ffac0104010 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffabf59e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac0109270 0x7ffac01a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffabf59e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac0109270 0x7ffac01a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48470/0 (socket says 172.21.15.122:48470) 2024-05-03T17:53:41.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffabf59e700 1 -- 172.21.15.122:0/291139083 learned_addr learned my addr 172.21.15.122:0/291139083 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffabe59c700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac0106940 0x7ffac0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.434+0000 7ffabed9d700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffac0104010 0x7ffac0199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.435+0000 7ffabf59e700 1 -- 172.21.15.122:0/291139083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffac0104010 msgr2=0x7ffac0199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.435+0000 7ffabf59e700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffac0104010 0x7ffac0199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.435+0000 7ffabf59e700 1 -- 172.21.15.122:0/291139083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac0106940 msgr2=0x7ffac0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.435+0000 7ffabf59e700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac0106940 0x7ffac0199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.435+0000 7ffabf59e700 1 -- 172.21.15.122:0/291139083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac01a11d0 con 0x7ffac0109270 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.435+0000 7ffabf59e700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac0109270 0x7ffac01a0b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffab800add0 tx=0x7ffab800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.435+0000 7ffaa7fff700 1 -- 172.21.15.122:0/291139083 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8017020 con 0x7ffac0109270 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.435+0000 7ffac50a7700 1 -- 172.21.15.122:0/291139083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac01a14c0 con 0x7ffac0109270 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.435+0000 7ffaa7fff700 1 -- 172.21.15.122:0/291139083 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffab800ae00 con 0x7ffac0109270 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.435+0000 7ffaa7fff700 1 -- 172.21.15.122:0/291139083 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8005280 con 0x7ffac0109270 2024-05-03T17:53:41.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.435+0000 7ffac50a7700 1 -- 172.21.15.122:0/291139083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac01a19b0 con 0x7ffac0109270 2024-05-03T17:53:41.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.436+0000 7ffac50a7700 1 -- 172.21.15.122:0/291139083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac00519e0 con 0x7ffac0109270 2024-05-03T17:53:41.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.437+0000 7ffaa7fff700 1 -- 172.21.15.122:0/291139083 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7ffab800c890 con 0x7ffac0109270 2024-05-03T17:53:41.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.438+0000 7ffaa7fff700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaa80b29b0 0x7ffaa80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:41.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.438+0000 7ffabed9d700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaa80b29b0 0x7ffaa80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:41.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.438+0000 7ffaa7fff700 1 -- 172.21.15.122:0/291139083 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7ffab80e1e20 con 0x7ffac0109270 2024-05-03T17:53:41.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.438+0000 7ffabed9d700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaa80b29b0 0x7ffaa80b4e70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffab0005950 tx=0x7ffab000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:41.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.441+0000 7ffaa7fff700 1 -- 172.21.15.122:0/291139083 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ffab8021020 con 0x7ffac0109270 2024-05-03T17:53:41.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.729+0000 7ffac50a7700 1 -- 172.21.15.122:0/291139083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7ffac00640b0 con 0x7ffac0109270 2024-05-03T17:53:41.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.732+0000 7ffaa7fff700 1 -- 172.21.15.122:0/291139083 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7ffab80a9b70 con 0x7ffac0109270 2024-05-03T17:53:41.727 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-05-03T17:53:41.727 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQClJDVm7vqTKxAATKNsyfu6OygSRzumJv5huQ== 2024-05-03T17:53:41.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 -- 172.21.15.122:0/291139083 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaa80b29b0 msgr2=0x7ffaa80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:41.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaa80b29b0 0x7ffaa80b4e70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffab0005950 tx=0x7ffab000b410 comp rx=0 tx=0).stop 2024-05-03T17:53:41.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 -- 172.21.15.122:0/291139083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac0109270 msgr2=0x7ffac01a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:41.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac0109270 0x7ffac01a0b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffab800add0 tx=0x7ffab800c330 comp rx=0 tx=0).stop 2024-05-03T17:53:41.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 -- 172.21.15.122:0/291139083 shutdown_connections 2024-05-03T17:53:41.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffaa80b29b0 0x7ffaa80b4e70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:41.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac0109270 0x7ffac01a0b30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:41.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffac0104010 0x7ffac0199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:41.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 --2- 172.21.15.122:0/291139083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffac0106940 0x7ffac0199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:41.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 -- 172.21.15.122:0/291139083 >> 172.21.15.122:0/291139083 conn(0x7ffac00fd9c0 msgr2=0x7ffac0104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:41.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 -- 172.21.15.122:0/291139083 shutdown_connections 2024-05-03T17:53:41.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:41.734+0000 7ffac50a7700 1 -- 172.21.15.122:0/291139083 wait complete. 2024-05-03T17:53:42.333 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:42 smithi122 ceph-mon[29744]: pgmap v320: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-03T17:53:42.333 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:42 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/291139083' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T17:53:42.333 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:42 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/291139083' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T17:53:42.360 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:53:42.360 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-03T17:53:42.360 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-03T17:53:42.397 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-03T17:53:42.397 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-03T17:53:42.397 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-03T17:53:42.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:42 smithi149 ceph-mon[33409]: pgmap v320: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-03T17:53:42.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:42 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/291139083' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T17:53:42.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:42 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/291139083' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:42 smithi192 ceph-mon[33753]: pgmap v320: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-03T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:42 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/291139083' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:42 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/291139083' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T17:53:42.671 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:44.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.378+0000 7fafe5a3c700 1 -- 172.21.15.122:0/2627884689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafe0103750 msgr2=0x7fafe0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:44.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.378+0000 7fafe5a3c700 1 --2- 172.21.15.122:0/2627884689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafe0103750 0x7fafe0103b30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fafd00099c0 tx=0x7fafd0009cd0 comp rx=0 tx=0).stop 2024-05-03T17:53:44.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.378+0000 7fafe5a3c700 1 -- 172.21.15.122:0/2627884689 shutdown_connections 2024-05-03T17:53:44.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.378+0000 7fafe5a3c700 1 --2- 172.21.15.122:0/2627884689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafe0108850 0x7fafe01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:44.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.378+0000 7fafe5a3c700 1 --2- 172.21.15.122:0/2627884689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafe0103750 0x7fafe0103b30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:44.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.378+0000 7fafe5a3c700 1 --2- 172.21.15.122:0/2627884689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafe0104100 0x7fafe0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:44.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.378+0000 7fafe5a3c700 1 -- 172.21.15.122:0/2627884689 >> 172.21.15.122:0/2627884689 conn(0x7fafe007a600 msgr2=0x7fafe007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:44.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.378+0000 7fafe5a3c700 1 -- 172.21.15.122:0/2627884689 shutdown_connections 2024-05-03T17:53:44.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.378+0000 7fafe5a3c700 1 -- 172.21.15.122:0/2627884689 wait complete. 2024-05-03T17:53:44.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.379+0000 7fafe5a3c700 1 Processor -- start 2024-05-03T17:53:44.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.379+0000 7fafe5a3c700 1 -- start start 2024-05-03T17:53:44.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.379+0000 7fafe5a3c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafe0103750 0x7fafe019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:44.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.379+0000 7fafe5a3c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafe0104100 0x7fafe019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:44.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.379+0000 7fafe5a3c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafe0108850 0x7fafe0199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:44.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.379+0000 7fafe5a3c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0113640 con 0x7fafe0108850 2024-05-03T17:53:44.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.379+0000 7fafe5a3c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01134c0 con 0x7fafe0104100 2024-05-03T17:53:44.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.379+0000 7fafe5a3c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01137c0 con 0x7fafe0103750 2024-05-03T17:53:44.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.379+0000 7fafdf7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafe0108850 0x7fafe0199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:44.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafdf7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafe0108850 0x7fafe0199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48486/0 (socket says 172.21.15.122:48486) 2024-05-03T17:53:44.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafdf7fe700 1 -- 172.21.15.122:0/3938490250 learned_addr learned my addr 172.21.15.122:0/3938490250 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:44.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafdeffd700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafe0103750 0x7fafe019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:44.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafde7fc700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafe0104100 0x7fafe019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:44.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafdf7fe700 1 -- 172.21.15.122:0/3938490250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafe0103750 msgr2=0x7fafe019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:44.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafdf7fe700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafe0103750 0x7fafe019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:44.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafdf7fe700 1 -- 172.21.15.122:0/3938490250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafe0104100 msgr2=0x7fafe019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:44.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafdf7fe700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafe0104100 0x7fafe019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:44.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafdf7fe700 1 -- 172.21.15.122:0/3938490250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe019a080 con 0x7fafe0108850 2024-05-03T17:53:44.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafdf7fe700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafe0108850 0x7fafe0199920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fafd8007ee0 tx=0x7fafd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:44.379 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:44 smithi122 ceph-mon[29744]: pgmap v321: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T17:53:44.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafe4a3a700 1 -- 172.21.15.122:0/3938490250 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8004bb0 con 0x7fafe0108850 2024-05-03T17:53:44.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafe5a3c700 1 -- 172.21.15.122:0/3938490250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe019a370 con 0x7fafe0108850 2024-05-03T17:53:44.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafe4a3a700 1 -- 172.21.15.122:0/3938490250 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafd8004d50 con 0x7fafe0108850 2024-05-03T17:53:44.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafe4a3a700 1 -- 172.21.15.122:0/3938490250 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8005920 con 0x7fafe0108850 2024-05-03T17:53:44.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.380+0000 7fafe5a3c700 1 -- 172.21.15.122:0/3938490250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe01a6630 con 0x7fafe0108850 2024-05-03T17:53:44.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.381+0000 7fafe5a3c700 1 -- 172.21.15.122:0/3938490250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe010ba90 con 0x7fafe0108850 2024-05-03T17:53:44.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.383+0000 7fafe4a3a700 1 -- 172.21.15.122:0/3938490250 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fafd801a020 con 0x7fafe0108850 2024-05-03T17:53:44.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.383+0000 7fafe4a3a700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafc80b2910 0x7fafc80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:44.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.383+0000 7fafdeffd700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafc80b2910 0x7fafc80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:44.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.383+0000 7fafe4a3a700 1 -- 172.21.15.122:0/3938490250 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fafd80e1f70 con 0x7fafe0108850 2024-05-03T17:53:44.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.384+0000 7fafdeffd700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafc80b2910 0x7fafc80b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fafe019aa60 tx=0x7fafd0005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:44.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.387+0000 7fafe4a3a700 1 -- 172.21.15.122:0/3938490250 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fafd8010070 con 0x7fafe0108850 2024-05-03T17:53:44.536 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:44 smithi149 ceph-mon[33409]: pgmap v321: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T17:53:44.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:44 smithi192 ceph-mon[33753]: pgmap v321: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T17:53:44.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.656+0000 7fafe5a3c700 1 -- 172.21.15.122:0/3938490250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fafe010b470 con 0x7fafe0108850 2024-05-03T17:53:44.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.661+0000 7fafe4a3a700 1 -- 172.21.15.122:0/3938490250 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288655 (secure 0 0 0) 0x7fafd80a9c30 con 0x7fafe0108850 2024-05-03T17:53:44.660 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:53:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 -- 172.21.15.122:0/3938490250 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafc80b2910 msgr2=0x7fafc80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafc80b2910 0x7fafc80b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fafe019aa60 tx=0x7fafd0005c40 comp rx=0 tx=0).stop 2024-05-03T17:53:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 -- 172.21.15.122:0/3938490250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafe0108850 msgr2=0x7fafe0199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafe0108850 0x7fafe0199920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fafd8007ee0 tx=0x7fafd800d3b0 comp rx=0 tx=0).stop 2024-05-03T17:53:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 -- 172.21.15.122:0/3938490250 shutdown_connections 2024-05-03T17:53:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafc80b2910 0x7fafc80b4dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafe0108850 0x7fafe0199920 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafe0103750 0x7fafe019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 --2- 172.21.15.122:0/3938490250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafe0104100 0x7fafe019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 -- 172.21.15.122:0/3938490250 >> 172.21.15.122:0/3938490250 conn(0x7fafe007a600 msgr2=0x7fafe0101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 -- 172.21.15.122:0/3938490250 shutdown_connections 2024-05-03T17:53:44.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:44.664+0000 7fafe5a3c700 1 -- 172.21.15.122:0/3938490250 wait complete. 2024-05-03T17:53:45.399 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1089873263},{"type":"v1","addr":"172.21.15.122:6801","nonce":1089873263}]},"active_addr":"172.21.15.122:6801/1089873263","active_change":"2024-05-03T17:45:18.454515+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.122:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":1748943908}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":1188558552}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":2501443656}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":2746493606}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":428926442}]}]} 2024-05-03T17:53:45.405 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-03T17:53:45.405 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-03T17:53:45.405 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T17:53:45.536 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:45 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3938490250' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T17:53:45.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:45 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3938490250' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T17:53:45.646 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:45.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:45 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3938490250' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T17:53:46.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:46 smithi122 ceph-mon[29744]: pgmap v322: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:53:46.536 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:46 smithi149 ceph-mon[33409]: pgmap v322: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:53:46.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:46 smithi192 ceph-mon[33753]: pgmap v322: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:53:47.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.282+0000 7f25607a3700 1 -- 172.21.15.122:0/2122251899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2558102d60 msgr2=0x7f255810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:47.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.282+0000 7f25607a3700 1 --2- 172.21.15.122:0/2122251899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2558102d60 0x7f255810f250 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f255000b0a0 tx=0x7f255000b3b0 comp rx=0 tx=0).stop 2024-05-03T17:53:47.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.282+0000 7f25607a3700 1 -- 172.21.15.122:0/2122251899 shutdown_connections 2024-05-03T17:53:47.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.282+0000 7f25607a3700 1 --2- 172.21.15.122:0/2122251899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2558102d60 0x7f255810f250 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:47.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.282+0000 7f25607a3700 1 --2- 172.21.15.122:0/2122251899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558108410 0x7f25581087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:47.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.282+0000 7f25607a3700 1 --2- 172.21.15.122:0/2122251899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25581023a0 0x7f2558102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:47.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.282+0000 7f25607a3700 1 -- 172.21.15.122:0/2122251899 >> 172.21.15.122:0/2122251899 conn(0x7f2558079350 msgr2=0x7f2558079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.282+0000 7f25607a3700 1 -- 172.21.15.122:0/2122251899 shutdown_connections 2024-05-03T17:53:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.282+0000 7f25607a3700 1 -- 172.21.15.122:0/2122251899 wait complete. 2024-05-03T17:53:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f25607a3700 1 Processor -- start 2024-05-03T17:53:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f25607a3700 1 -- start start 2024-05-03T17:53:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f25607a3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25581023a0 0x7f255819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f25607a3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2558102d60 0x7f255819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f25607a3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558108410 0x7f25581a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f25607a3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2558111490 con 0x7f25581023a0 2024-05-03T17:53:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f25607a3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2558111310 con 0x7f2558102d60 2024-05-03T17:53:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f25607a3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2558111610 con 0x7f2558108410 2024-05-03T17:53:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f255e53f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25581023a0 0x7f255819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f255e53f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25581023a0 0x7f255819bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48508/0 (socket says 172.21.15.122:48508) 2024-05-03T17:53:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f255e53f700 1 -- 172.21.15.122:0/3140738570 learned_addr learned my addr 172.21.15.122:0/3140738570 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f255dd3e700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2558102d60 0x7f255819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.283+0000 7f255ed40700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558108410 0x7f25581a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.284+0000 7f255e53f700 1 -- 172.21.15.122:0/3140738570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558108410 msgr2=0x7f25581a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.284+0000 7f255e53f700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558108410 0x7f25581a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.284+0000 7f255e53f700 1 -- 172.21.15.122:0/3140738570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2558102d60 msgr2=0x7f255819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.284+0000 7f255e53f700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2558102d60 0x7f255819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.284+0000 7f255e53f700 1 -- 172.21.15.122:0/3140738570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25581a0d60 con 0x7f25581023a0 2024-05-03T17:53:47.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.284+0000 7f255e53f700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25581023a0 0x7f255819bd50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f255400d980 tx=0x7f255400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:47.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.284+0000 7f254f7fe700 1 -- 172.21.15.122:0/3140738570 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554017070 con 0x7f25581023a0 2024-05-03T17:53:47.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.284+0000 7f25607a3700 1 -- 172.21.15.122:0/3140738570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25581a1050 con 0x7f25581023a0 2024-05-03T17:53:47.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.284+0000 7f254f7fe700 1 -- 172.21.15.122:0/3140738570 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2554005020 con 0x7f25581023a0 2024-05-03T17:53:47.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.284+0000 7f254f7fe700 1 -- 172.21.15.122:0/3140738570 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554009b50 con 0x7f25581023a0 2024-05-03T17:53:47.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.284+0000 7f25607a3700 1 -- 172.21.15.122:0/3140738570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25581a88d0 con 0x7f25581023a0 2024-05-03T17:53:47.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.285+0000 7f25607a3700 1 -- 172.21.15.122:0/3140738570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f255806a190 con 0x7f25581023a0 2024-05-03T17:53:47.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.286+0000 7f254f7fe700 1 -- 172.21.15.122:0/3140738570 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f25540051d0 con 0x7f25581023a0 2024-05-03T17:53:47.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.287+0000 7f254f7fe700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25440b2910 0x7f25440b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:47.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.287+0000 7f255dd3e700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25440b2910 0x7f25440b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:47.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.287+0000 7f254f7fe700 1 -- 172.21.15.122:0/3140738570 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f25540e25b0 con 0x7f25581023a0 2024-05-03T17:53:47.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.287+0000 7f255dd3e700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25440b2910 0x7f25440b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2558079a90 tx=0x7f254800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:47.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.290+0000 7f254f7fe700 1 -- 172.21.15.122:0/3140738570 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2554020020 con 0x7f25581023a0 2024-05-03T17:53:47.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.499+0000 7f25607a3700 1 -- 172.21.15.122:0/3140738570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25581a8aa0 con 0x7f25581023a0 2024-05-03T17:53:47.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.501+0000 7f254f7fe700 1 -- 172.21.15.122:0/3140738570 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22622 (secure 0 0 0) 0x7f25540cae20 con 0x7f25581023a0 2024-05-03T17:53:47.496 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:53:47.496 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":79,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","created":"2024-05-03T17:44:17.415571+0000","modified":"2024-05-03T17:53:19.863832+0000","last_up_change":"2024-05-03T17:51:28.839117+0000","last_in_change":"2024-05-03T17:51:07.954070+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T17:47:40.539967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T17:52:31.646919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T17:52:34.503067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4b08628-0a29-44aa-b8b9-ed9657f12be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6803","nonce":1329364610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6805","nonce":1329364610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6809","nonce":1329364610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6807","nonce":1329364610}]},"public_addr":"172.21.15.122:6803/1329364610","cluster_addr":"172.21.15.122:6805/1329364610","heartbeat_back_addr":"172.21.15.122:6809/1329364610","heartbeat_front_addr":"172.21.15.122:6807/1329364610","state":["exists","up"]},{"osd":1,"uuid":"ba550787-ce76-4944-8ebd-97e19590bb3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6801","nonce":1402463302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6803","nonce":1402463302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6807","nonce":1402463302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6805","nonce":1402463302}]},"public_addr":"172.21.15.149:6801/1402463302","cluster_addr":"172.21.15.149:6803/1402463302","heartbeat_back_addr":"172.21.15.149:6807/1402463302","heartbeat_front_addr":"172.21.15.149:6805/1402463302","state":["exists","up"]},{"osd":2,"uuid":"35ae7c47-d710-47c5-b0ec-91fa12b5be07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6801","nonce":3757191110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6803","nonce":3757191110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6807","nonce":3757191110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6805","nonce":3757191110}]},"public_addr":"172.21.15.192:6801/3757191110","cluster_addr":"172.21.15.192:6803/3757191110","heartbeat_back_addr":"172.21.15.192:6807/3757191110","heartbeat_front_addr":"172.21.15.192:6805/3757191110","state":["exists","up"]},{"osd":3,"uuid":"9973c0c1-ae73-497a-845e-664a20666df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6811","nonce":2359143764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6813","nonce":2359143764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6817","nonce":2359143764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6815","nonce":2359143764}]},"public_addr":"172.21.15.122:6811/2359143764","cluster_addr":"172.21.15.122:6813/2359143764","heartbeat_back_addr":"172.21.15.122:6817/2359143764","heartbeat_front_addr":"172.21.15.122:6815/2359143764","state":["exists","up"]},{"osd":4,"uuid":"da71dc18-94b9-4f7d-a061-7cc7b77db282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3627037532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3627037532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3627037532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3627037532}]},"public_addr":"172.21.15.149:6809/3627037532","cluster_addr":"172.21.15.149:6811/3627037532","heartbeat_back_addr":"172.21.15.149:6815/3627037532","heartbeat_front_addr":"172.21.15.149:6813/3627037532","state":["exists","up"]},{"osd":5,"uuid":"bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6809","nonce":455779485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6811","nonce":455779485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6815","nonce":455779485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6813","nonce":455779485}]},"public_addr":"172.21.15.192:6809/455779485","cluster_addr":"172.21.15.192:6811/455779485","heartbeat_back_addr":"172.21.15.192:6815/455779485","heartbeat_front_addr":"172.21.15.192:6813/455779485","state":["exists","up"]},{"osd":6,"uuid":"4150bfe5-a0e4-4bae-914a-75598f017519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6819","nonce":3419421998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6821","nonce":3419421998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6825","nonce":3419421998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6823","nonce":3419421998}]},"public_addr":"172.21.15.122:6819/3419421998","cluster_addr":"172.21.15.122:6821/3419421998","heartbeat_back_addr":"172.21.15.122:6825/3419421998","heartbeat_front_addr":"172.21.15.122:6823/3419421998","state":["exists","up"]},{"osd":7,"uuid":"71e01812-e511-44a0-a143-2afca510ae8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6817","nonce":139747549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6819","nonce":139747549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6823","nonce":139747549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6821","nonce":139747549}]},"public_addr":"172.21.15.149:6817/139747549","cluster_addr":"172.21.15.149:6819/139747549","heartbeat_back_addr":"172.21.15.149:6823/139747549","heartbeat_front_addr":"172.21.15.149:6821/139747549","state":["exists","up"]},{"osd":8,"uuid":"5294516d-fac6-41d0-a635-6bc1d7eca585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6817","nonce":2080816744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6819","nonce":2080816744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6823","nonce":2080816744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6821","nonce":2080816744}]},"public_addr":"172.21.15.192:6817/2080816744","cluster_addr":"172.21.15.192:6819/2080816744","heartbeat_back_addr":"172.21.15.192:6823/2080816744","heartbeat_front_addr":"172.21.15.192:6821/2080816744","state":["exists","up"]},{"osd":9,"uuid":"f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6827","nonce":1773691421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6829","nonce":1773691421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6833","nonce":1773691421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6831","nonce":1773691421}]},"public_addr":"172.21.15.122:6827/1773691421","cluster_addr":"172.21.15.122:6829/1773691421","heartbeat_back_addr":"172.21.15.122:6833/1773691421","heartbeat_front_addr":"172.21.15.122:6831/1773691421","state":["exists","up"]},{"osd":10,"uuid":"af88e50f-98f1-460d-856b-a920dd2ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6825","nonce":1954186137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6827","nonce":1954186137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6831","nonce":1954186137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6829","nonce":1954186137}]},"public_addr":"172.21.15.149:6825/1954186137","cluster_addr":"172.21.15.149:6827/1954186137","heartbeat_back_addr":"172.21.15.149:6831/1954186137","heartbeat_front_addr":"172.21.15.149:6829/1954186137","state":["exists","up"]},{"osd":11,"uuid":"b54c1b4d-cdb5-42d9-979a-47f9a870b5cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6825","nonce":2098474209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6827","nonce":2098474209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6831","nonce":2098474209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6829","nonce":2098474209}]},"public_addr":"172.21.15.192:6825/2098474209","cluster_addr":"172.21.15.192:6827/2098474209","heartbeat_back_addr":"172.21.15.192:6831/2098474209","heartbeat_front_addr":"172.21.15.192:6829/2098474209","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:46:50.049769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:13.518485+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:37.473023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:01.809640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:26.648514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:51.268056+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:16.294834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:42.079065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:07.609271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:33.885127+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:59.982708+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:51:26.461369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/2089907395":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6801/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:0/2001860625":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/1888284988":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/256574472":"2024-05-04T17:44:52.244049+0000","172.21.15.122:6801/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/64546794":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/584846864":"2024-05-04T17:45:18.454406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T17:53:47.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 -- 172.21.15.122:0/3140738570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25440b2910 msgr2=0x7f25440b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:47.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25440b2910 0x7f25440b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2558079a90 tx=0x7f254800b410 comp rx=0 tx=0).stop 2024-05-03T17:53:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 -- 172.21.15.122:0/3140738570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25581023a0 msgr2=0x7f255819bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25581023a0 0x7f255819bd50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f255400d980 tx=0x7f255400dc90 comp rx=0 tx=0).stop 2024-05-03T17:53:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 -- 172.21.15.122:0/3140738570 shutdown_connections 2024-05-03T17:53:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25440b2910 0x7f25440b4dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2558108410 0x7f25581a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25581023a0 0x7f255819bd50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 --2- 172.21.15.122:0/3140738570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2558102d60 0x7f255819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 -- 172.21.15.122:0/3140738570 >> 172.21.15.122:0/3140738570 conn(0x7f2558079350 msgr2=0x7f2558105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 -- 172.21.15.122:0/3140738570 shutdown_connections 2024-05-03T17:53:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:47.503+0000 7f25607a3700 1 -- 172.21.15.122:0/3140738570 wait complete. 2024-05-03T17:53:48.170 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-03T17:53:48.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T17:53:48.410 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:48.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:48 smithi122 ceph-mon[29744]: pgmap v323: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:53:48.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:48 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3140738570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:53:48.536 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:48 smithi149 ceph-mon[33409]: pgmap v323: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:53:48.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:48 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3140738570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:53:48.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:48 smithi192 ceph-mon[33753]: pgmap v323: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:53:48.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:48 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3140738570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:53:50.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.105+0000 7f56ac55c700 1 -- 172.21.15.122:0/1075905037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a4108890 msgr2=0x7f56a4110410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:50.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.105+0000 7f56ac55c700 1 --2- 172.21.15.122:0/1075905037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a4108890 0x7f56a4110410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f569c00b0d0 tx=0x7f569c00b3e0 comp rx=0 tx=0).stop 2024-05-03T17:53:50.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.105+0000 7f56ac55c700 1 -- 172.21.15.122:0/1075905037 shutdown_connections 2024-05-03T17:53:50.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.105+0000 7f56ac55c700 1 --2- 172.21.15.122:0/1075905037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a4108890 0x7f56a4110410 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:50.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.105+0000 7f56ac55c700 1 --2- 172.21.15.122:0/1075905037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a4101c50 0x7f56a4102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:50.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.105+0000 7f56ac55c700 1 --2- 172.21.15.122:0/1075905037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56a4102600 0x7f56a4108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:50.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.105+0000 7f56ac55c700 1 -- 172.21.15.122:0/1075905037 >> 172.21.15.122:0/1075905037 conn(0x7f56a40789a0 msgr2=0x7f56a40ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:50.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.105+0000 7f56ac55c700 1 -- 172.21.15.122:0/1075905037 shutdown_connections 2024-05-03T17:53:50.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.106+0000 7f56ac55c700 1 -- 172.21.15.122:0/1075905037 wait complete. 2024-05-03T17:53:50.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.106+0000 7f56ac55c700 1 Processor -- start 2024-05-03T17:53:50.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.106+0000 7f56ac55c700 1 -- start start 2024-05-03T17:53:50.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.106+0000 7f56ac55c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a4101c50 0x7f56a419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:50.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.106+0000 7f56ac55c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56a4102600 0x7f56a419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:50.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.106+0000 7f56ac55c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a4108890 0x7f56a4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:50.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56ac55c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41136d0 con 0x7f56a4101c50 2024-05-03T17:53:50.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56ac55c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4113550 con 0x7f56a4108890 2024-05-03T17:53:50.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56ac55c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4113850 con 0x7f56a4102600 2024-05-03T17:53:50.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56aa2f8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a4101c50 0x7f56a419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:50.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56aa2f8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a4101c50 0x7f56a419f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52332/0 (socket says 172.21.15.122:52332) 2024-05-03T17:53:50.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56aa2f8700 1 -- 172.21.15.122:0/2911166502 learned_addr learned my addr 172.21.15.122:0/2911166502 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:50.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56aaaf9700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a4108890 0x7f56a4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:50.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56a9af7700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56a4102600 0x7f56a419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:50.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56aa2f8700 1 -- 172.21.15.122:0/2911166502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56a4102600 msgr2=0x7f56a419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:50.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56aa2f8700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56a4102600 0x7f56a419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:50.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56aa2f8700 1 -- 172.21.15.122:0/2911166502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a4108890 msgr2=0x7f56a4199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:50.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56aa2f8700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a4108890 0x7f56a4199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:50.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56aa2f8700 1 -- 172.21.15.122:0/2911166502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a419a0b0 con 0x7f56a4101c50 2024-05-03T17:53:50.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56aa2f8700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a4101c50 0x7f56a419f740 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f56a000d980 tx=0x7f56a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:50.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f569b7fe700 1 -- 172.21.15.122:0/2911166502 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0016070 con 0x7f56a4101c50 2024-05-03T17:53:50.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f569b7fe700 1 -- 172.21.15.122:0/2911166502 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a0004830 con 0x7f56a4101c50 2024-05-03T17:53:50.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56ac55c700 1 -- 172.21.15.122:0/2911166502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a419a3a0 con 0x7f56a4101c50 2024-05-03T17:53:50.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f569b7fe700 1 -- 172.21.15.122:0/2911166502 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a00054c0 con 0x7f56a4101c50 2024-05-03T17:53:50.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.107+0000 7f56ac55c700 1 -- 172.21.15.122:0/2911166502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a41a6610 con 0x7f56a4101c50 2024-05-03T17:53:50.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.108+0000 7f56ac55c700 1 -- 172.21.15.122:0/2911166502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a406a190 con 0x7f56a4101c50 2024-05-03T17:53:50.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.110+0000 7f569b7fe700 1 -- 172.21.15.122:0/2911166502 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f56a0004220 con 0x7f56a4101c50 2024-05-03T17:53:50.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.111+0000 7f569b7fe700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f56900b2990 0x7f56900b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:50.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.111+0000 7f56a9af7700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f56900b2990 0x7f56900b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:50.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.111+0000 7f569b7fe700 1 -- 172.21.15.122:0/2911166502 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f56a00e25b0 con 0x7f56a4101c50 2024-05-03T17:53:50.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.111+0000 7f56a9af7700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f56900b2990 0x7f56900b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5694006fd0 tx=0x7f5694008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:50.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.114+0000 7f569b7fe700 1 -- 172.21.15.122:0/2911166502 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f56a001f020 con 0x7f56a4101c50 2024-05-03T17:53:50.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.326+0000 7f56ac55c700 1 -- 172.21.15.122:0/2911166502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56a402e510 con 0x7f56a4101c50 2024-05-03T17:53:50.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.327+0000 7f569b7fe700 1 -- 172.21.15.122:0/2911166502 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22622 (secure 0 0 0) 0x7f56a00cae20 con 0x7f56a4101c50 2024-05-03T17:53:50.323 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:53:50.323 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":79,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","created":"2024-05-03T17:44:17.415571+0000","modified":"2024-05-03T17:53:19.863832+0000","last_up_change":"2024-05-03T17:51:28.839117+0000","last_in_change":"2024-05-03T17:51:07.954070+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T17:47:40.539967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T17:52:31.646919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T17:52:34.503067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4b08628-0a29-44aa-b8b9-ed9657f12be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6803","nonce":1329364610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6805","nonce":1329364610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6809","nonce":1329364610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6807","nonce":1329364610}]},"public_addr":"172.21.15.122:6803/1329364610","cluster_addr":"172.21.15.122:6805/1329364610","heartbeat_back_addr":"172.21.15.122:6809/1329364610","heartbeat_front_addr":"172.21.15.122:6807/1329364610","state":["exists","up"]},{"osd":1,"uuid":"ba550787-ce76-4944-8ebd-97e19590bb3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6801","nonce":1402463302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6803","nonce":1402463302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6807","nonce":1402463302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6805","nonce":1402463302}]},"public_addr":"172.21.15.149:6801/1402463302","cluster_addr":"172.21.15.149:6803/1402463302","heartbeat_back_addr":"172.21.15.149:6807/1402463302","heartbeat_front_addr":"172.21.15.149:6805/1402463302","state":["exists","up"]},{"osd":2,"uuid":"35ae7c47-d710-47c5-b0ec-91fa12b5be07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6801","nonce":3757191110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6803","nonce":3757191110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6807","nonce":3757191110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6805","nonce":3757191110}]},"public_addr":"172.21.15.192:6801/3757191110","cluster_addr":"172.21.15.192:6803/3757191110","heartbeat_back_addr":"172.21.15.192:6807/3757191110","heartbeat_front_addr":"172.21.15.192:6805/3757191110","state":["exists","up"]},{"osd":3,"uuid":"9973c0c1-ae73-497a-845e-664a20666df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6811","nonce":2359143764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6813","nonce":2359143764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6817","nonce":2359143764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6815","nonce":2359143764}]},"public_addr":"172.21.15.122:6811/2359143764","cluster_addr":"172.21.15.122:6813/2359143764","heartbeat_back_addr":"172.21.15.122:6817/2359143764","heartbeat_front_addr":"172.21.15.122:6815/2359143764","state":["exists","up"]},{"osd":4,"uuid":"da71dc18-94b9-4f7d-a061-7cc7b77db282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3627037532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3627037532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3627037532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3627037532}]},"public_addr":"172.21.15.149:6809/3627037532","cluster_addr":"172.21.15.149:6811/3627037532","heartbeat_back_addr":"172.21.15.149:6815/3627037532","heartbeat_front_addr":"172.21.15.149:6813/3627037532","state":["exists","up"]},{"osd":5,"uuid":"bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6809","nonce":455779485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6811","nonce":455779485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6815","nonce":455779485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6813","nonce":455779485}]},"public_addr":"172.21.15.192:6809/455779485","cluster_addr":"172.21.15.192:6811/455779485","heartbeat_back_addr":"172.21.15.192:6815/455779485","heartbeat_front_addr":"172.21.15.192:6813/455779485","state":["exists","up"]},{"osd":6,"uuid":"4150bfe5-a0e4-4bae-914a-75598f017519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6819","nonce":3419421998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6821","nonce":3419421998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6825","nonce":3419421998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6823","nonce":3419421998}]},"public_addr":"172.21.15.122:6819/3419421998","cluster_addr":"172.21.15.122:6821/3419421998","heartbeat_back_addr":"172.21.15.122:6825/3419421998","heartbeat_front_addr":"172.21.15.122:6823/3419421998","state":["exists","up"]},{"osd":7,"uuid":"71e01812-e511-44a0-a143-2afca510ae8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6817","nonce":139747549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6819","nonce":139747549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6823","nonce":139747549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6821","nonce":139747549}]},"public_addr":"172.21.15.149:6817/139747549","cluster_addr":"172.21.15.149:6819/139747549","heartbeat_back_addr":"172.21.15.149:6823/139747549","heartbeat_front_addr":"172.21.15.149:6821/139747549","state":["exists","up"]},{"osd":8,"uuid":"5294516d-fac6-41d0-a635-6bc1d7eca585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6817","nonce":2080816744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6819","nonce":2080816744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6823","nonce":2080816744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6821","nonce":2080816744}]},"public_addr":"172.21.15.192:6817/2080816744","cluster_addr":"172.21.15.192:6819/2080816744","heartbeat_back_addr":"172.21.15.192:6823/2080816744","heartbeat_front_addr":"172.21.15.192:6821/2080816744","state":["exists","up"]},{"osd":9,"uuid":"f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6827","nonce":1773691421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6829","nonce":1773691421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6833","nonce":1773691421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6831","nonce":1773691421}]},"public_addr":"172.21.15.122:6827/1773691421","cluster_addr":"172.21.15.122:6829/1773691421","heartbeat_back_addr":"172.21.15.122:6833/1773691421","heartbeat_front_addr":"172.21.15.122:6831/1773691421","state":["exists","up"]},{"osd":10,"uuid":"af88e50f-98f1-460d-856b-a920dd2ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6825","nonce":1954186137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6827","nonce":1954186137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6831","nonce":1954186137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6829","nonce":1954186137}]},"public_addr":"172.21.15.149:6825/1954186137","cluster_addr":"172.21.15.149:6827/1954186137","heartbeat_back_addr":"172.21.15.149:6831/1954186137","heartbeat_front_addr":"172.21.15.149:6829/1954186137","state":["exists","up"]},{"osd":11,"uuid":"b54c1b4d-cdb5-42d9-979a-47f9a870b5cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6825","nonce":2098474209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6827","nonce":2098474209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6831","nonce":2098474209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6829","nonce":2098474209}]},"public_addr":"172.21.15.192:6825/2098474209","cluster_addr":"172.21.15.192:6827/2098474209","heartbeat_back_addr":"172.21.15.192:6831/2098474209","heartbeat_front_addr":"172.21.15.192:6829/2098474209","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:46:50.049769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:13.518485+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:37.473023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:01.809640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:26.648514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:51.268056+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:16.294834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:42.079065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:07.609271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:33.885127+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:59.982708+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:51:26.461369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/2089907395":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6801/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:0/2001860625":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/1888284988":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/256574472":"2024-05-04T17:44:52.244049+0000","172.21.15.122:6801/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/64546794":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/584846864":"2024-05-04T17:45:18.454406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T17:53:50.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.329+0000 7f56ac55c700 1 -- 172.21.15.122:0/2911166502 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f56900b2990 msgr2=0x7f56900b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:50.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.329+0000 7f56ac55c700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f56900b2990 0x7f56900b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5694006fd0 tx=0x7f5694008040 comp rx=0 tx=0).stop 2024-05-03T17:53:50.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.329+0000 7f56ac55c700 1 -- 172.21.15.122:0/2911166502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a4101c50 msgr2=0x7f56a419f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:50.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.329+0000 7f56ac55c700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a4101c50 0x7f56a419f740 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f56a000d980 tx=0x7f56a000dc90 comp rx=0 tx=0).stop 2024-05-03T17:53:50.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.329+0000 7f56ac55c700 1 -- 172.21.15.122:0/2911166502 shutdown_connections 2024-05-03T17:53:50.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.329+0000 7f56ac55c700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f56900b2990 0x7f56900b4e50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:50.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.329+0000 7f56ac55c700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a4108890 0x7f56a4199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:50.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.329+0000 7f56ac55c700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a4101c50 0x7f56a419f740 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:50.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.329+0000 7f56ac55c700 1 --2- 172.21.15.122:0/2911166502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56a4102600 0x7f56a419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:50.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.329+0000 7f56ac55c700 1 -- 172.21.15.122:0/2911166502 >> 172.21.15.122:0/2911166502 conn(0x7f56a40789a0 msgr2=0x7f56a4107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:50.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.330+0000 7f56ac55c700 1 -- 172.21.15.122:0/2911166502 shutdown_connections 2024-05-03T17:53:50.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:50.330+0000 7f56ac55c700 1 -- 172.21.15.122:0/2911166502 wait complete. 2024-05-03T17:53:50.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:50 smithi122 ceph-mon[29744]: pgmap v324: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:53:50.536 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:50 smithi149 ceph-mon[33409]: pgmap v324: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:53:50.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:50 smithi192 ceph-mon[33753]: pgmap v324: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:53:51.010 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-03T17:53:51.011 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-03T17:53:51.011 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-03T17:53:51.012 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-03T17:53:51.012 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-03T17:53:51.012 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-03T17:53:51.013 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-03T17:53:51.013 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-03T17:53:51.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-03T17:53:51.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-03T17:53:51.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-03T17:53:51.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-03T17:53:51.293 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:51 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2911166502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:53:51.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:51 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2911166502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:53:51.671 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:51.681 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:51.685 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:51.756 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:51.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:51 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2911166502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:53:52.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:52 smithi122 ceph-mon[29744]: pgmap v325: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:53:52.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:52 smithi192 ceph-mon[33753]: pgmap v325: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:53:52.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:52 smithi149 ceph-mon[33409]: pgmap v325: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:53:54.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:54 smithi122 ceph-mon[29744]: pgmap v326: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:53:54.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:54 smithi192 ceph-mon[33753]: pgmap v326: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:53:54.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:54 smithi149 ceph-mon[33409]: pgmap v326: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:53:55.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.749+0000 7fb6aee8e700 1 -- 172.21.15.122:0/2066527329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a806c530 msgr2=0x7fb6a8110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:55.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.749+0000 7fb6aee8e700 1 --2- 172.21.15.122:0/2066527329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a806c530 0x7fb6a8110350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000b0a0 tx=0x7fb6a000b3b0 comp rx=0 tx=0).stop 2024-05-03T17:53:55.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.750+0000 7fb6aee8e700 1 -- 172.21.15.122:0/2066527329 shutdown_connections 2024-05-03T17:53:55.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.750+0000 7fb6aee8e700 1 --2- 172.21.15.122:0/2066527329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a806c530 0x7fb6a8110350 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:55.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.750+0000 7fb6aee8e700 1 --2- 172.21.15.122:0/2066527329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6a8109530 0x7fb6a8109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:55.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.750+0000 7fb6aee8e700 1 --2- 172.21.15.122:0/2066527329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6a806bb70 0x7fb6a806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:55.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.750+0000 7fb6aee8e700 1 -- 172.21.15.122:0/2066527329 >> 172.21.15.122:0/2066527329 conn(0x7fb6a8079350 msgr2=0x7fb6a8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:55.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.750+0000 7fb6aee8e700 1 -- 172.21.15.122:0/2066527329 shutdown_connections 2024-05-03T17:53:55.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.751+0000 7fb6aee8e700 1 -- 172.21.15.122:0/2066527329 wait complete. 2024-05-03T17:53:55.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.751+0000 7fb6aee8e700 1 Processor -- start 2024-05-03T17:53:55.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6aee8e700 1 -- start start 2024-05-03T17:53:55.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6aee8e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a806bb70 0x7fb6a8104440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:55.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6aee8e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6a806c530 0x7fb6a8102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:55.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6aee8e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6a8109530 0x7fb6a8102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:55.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6aee8e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a81136b0 con 0x7fb6a806bb70 2024-05-03T17:53:55.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6aee8e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a8113530 con 0x7fb6a8109530 2024-05-03T17:53:55.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6aee8e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a8113830 con 0x7fb6a806c530 2024-05-03T17:53:55.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6a7fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6a806c530 0x7fb6a8102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:55.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6acc2a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a806bb70 0x7fb6a8104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:55.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6a7fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6a806c530 0x7fb6a8102850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.122:45886/0 (socket says 172.21.15.122:45886) 2024-05-03T17:53:55.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6a7fff700 1 -- 172.21.15.122:0/1975269632 learned_addr learned my addr 172.21.15.122:0/1975269632 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:53:55.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6acc2a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a806bb70 0x7fb6a8104440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52362/0 (socket says 172.21.15.122:52362) 2024-05-03T17:53:55.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.752+0000 7fb6ad42b700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6a8109530 0x7fb6a8102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:55.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.753+0000 7fb6acc2a700 1 -- 172.21.15.122:0/1975269632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6a806c530 msgr2=0x7fb6a8102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:55.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.753+0000 7fb6acc2a700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6a806c530 0x7fb6a8102850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:55.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.753+0000 7fb6acc2a700 1 -- 172.21.15.122:0/1975269632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6a8109530 msgr2=0x7fb6a8102eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:55.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.753+0000 7fb6acc2a700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6a8109530 0x7fb6a8102eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:55.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.753+0000 7fb6acc2a700 1 -- 172.21.15.122:0/1975269632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a81ac530 con 0x7fb6a806bb70 2024-05-03T17:53:55.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.753+0000 7fb6acc2a700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a806bb70 0x7fb6a8104440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00c970 tx=0x7fb69c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:55.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.754+0000 7fb6a5ffb700 1 -- 172.21.15.122:0/1975269632 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00ce60 con 0x7fb6a806bb70 2024-05-03T17:53:55.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.754+0000 7fb6a5ffb700 1 -- 172.21.15.122:0/1975269632 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb69c004830 con 0x7fb6a806bb70 2024-05-03T17:53:55.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.754+0000 7fb6a5ffb700 1 -- 172.21.15.122:0/1975269632 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c005580 con 0x7fb6a806bb70 2024-05-03T17:53:55.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.756+0000 7fb6aee8e700 1 -- 172.21.15.122:0/1975269632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a81ac7c0 con 0x7fb6a806bb70 2024-05-03T17:53:55.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.756+0000 7fb6aee8e700 1 -- 172.21.15.122:0/1975269632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a81acc80 con 0x7fb6a806bb70 2024-05-03T17:53:55.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.757+0000 7fb6aee8e700 1 -- 172.21.15.122:0/1975269632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb6a8063bb0 con 0x7fb6a806bb70 2024-05-03T17:53:55.754 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:55.755 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:55.755 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:55.756 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:55.757 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:55.757 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:55.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.763+0000 7fb6a5ffb700 1 -- 172.21.15.122:0/1975269632 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb69c004d00 con 0x7fb6a806bb70 2024-05-03T17:53:55.759 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:53:55.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.764+0000 7fb6a5ffb700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb6980b2a60 0x7fb6980b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:55.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.765+0000 7fb6a5ffb700 1 -- 172.21.15.122:0/1975269632 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fb69c0e2530 con 0x7fb6a806bb70 2024-05-03T17:53:55.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.765+0000 7fb6a5ffb700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485] conn(0x7fb6980bc4a0 0x7fb6980be940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:53:55.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.765+0000 7fb6a5ffb700 1 -- 172.21.15.122:0/1975269632 --> [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb69c023e50 con 0x7fb6980bc4a0 2024-05-03T17:53:55.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.765+0000 7fb6a5ffb700 1 -- 172.21.15.122:0/1975269632 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fb69c0e2a10 con 0x7fb6a806bb70 2024-05-03T17:53:55.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.766+0000 7fb6a7fff700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb6980b2a60 0x7fb6980b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:55.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.767+0000 7fb6ad42b700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485] conn(0x7fb6980bc4a0 0x7fb6980be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:53:55.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.767+0000 7fb6a7fff700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb6980b2a60 0x7fb6980b4f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8103fc0 tx=0x7fb69400b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:55.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.768+0000 7fb6ad42b700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485] conn(0x7fb6980bc4a0 0x7fb6980be940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:53:55.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.770+0000 7fb6a5ffb700 1 -- 172.21.15.122:0/1975269632 <== osd.5 v2:172.21.15.192:6808/455779485 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb69c023e50 con 0x7fb6980bc4a0 2024-05-03T17:53:55.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.803+0000 7fb6aee8e700 1 -- 172.21.15.122:0/1975269632 --> [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb6a8064080 con 0x7fb6980bc4a0 2024-05-03T17:53:55.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.804+0000 7fb6a5ffb700 1 -- 172.21.15.122:0/1975269632 <== osd.5 v2:172.21.15.192:6808/455779485 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb6a8064080 con 0x7fb6980bc4a0 2024-05-03T17:53:55.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.804+0000 7fb6aee8e700 1 -- 172.21.15.122:0/1975269632 >> [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485] conn(0x7fb6980bc4a0 msgr2=0x7fb6980be940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.804+0000 7fb6aee8e700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485] conn(0x7fb6980bc4a0 0x7fb6980be940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:53:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.804+0000 7fb6aee8e700 1 -- 172.21.15.122:0/1975269632 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb6980b2a60 msgr2=0x7fb6980b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.804+0000 7fb6aee8e700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb6980b2a60 0x7fb6980b4f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8103fc0 tx=0x7fb69400b480 comp rx=0 tx=0).stop 2024-05-03T17:53:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.804+0000 7fb6aee8e700 1 -- 172.21.15.122:0/1975269632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a806bb70 msgr2=0x7fb6a8104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:53:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.804+0000 7fb6aee8e700 1 --2- 172.21.15.122:0/1975269632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a806bb70 0x7fb6a8104440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00c970 tx=0x7fb69c00cc80 comp rx=0 tx=0).stop 2024-05-03T17:53:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.804+0000 7fb6ad42b700 1 -- 172.21.15.122:0/1975269632 reap_dead start 2024-05-03T17:53:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.804+0000 7fb6aee8e700 1 -- 172.21.15.122:0/1975269632 shutdown_connections 2024-05-03T17:53:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.804+0000 7fb6aee8e700 1 -- 172.21.15.122:0/1975269632 >> 172.21.15.122:0/1975269632 conn(0x7fb6a8079350 msgr2=0x7fb6a81014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:53:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.805+0000 7fb6aee8e700 1 -- 172.21.15.122:0/1975269632 shutdown_connections 2024-05-03T17:53:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:53:55.805+0000 7fb6aee8e700 1 -- 172.21.15.122:0/1975269632 wait complete. 2024-05-03T17:53:56.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:56 smithi192 ceph-mon[33753]: pgmap v327: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:53:56.687 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:56 smithi122 ceph-mon[29744]: pgmap v327: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:53:56.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:56 smithi149 ceph-mon[33409]: pgmap v327: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:53:58.586 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:53:58 smithi122 ceph-mon[29744]: pgmap v328: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:53:58.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:53:58 smithi192 ceph-mon[33753]: pgmap v328: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:53:58.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:53:58 smithi149 ceph-mon[33409]: pgmap v328: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:00.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:00 smithi192 ceph-mon[33753]: pgmap v329: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:00.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:00 smithi122 ceph-mon[29744]: pgmap v329: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:00.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:00 smithi149 ceph-mon[33409]: pgmap v329: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:01.737 INFO:teuthology.orchestra.run.smithi122.stdout:154618822720 2024-05-03T17:54:01.737 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-03T17:54:02.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:02 smithi192 ceph-mon[33753]: pgmap v330: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:02.669 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:02 smithi122 ceph-mon[29744]: pgmap v330: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:02.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:02 smithi149 ceph-mon[33409]: pgmap v330: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:04.386 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:04 smithi122 ceph-mon[29744]: pgmap v331: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:04.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:04 smithi192 ceph-mon[33753]: pgmap v331: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:04.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.632+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/2092469620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b98103f00 msgr2=0x7f8b981103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:04.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.632+0000 7f8b9ff7e700 1 --2- 172.21.15.122:0/2092469620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b98103f00 0x7f8b981103b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000b0d0 tx=0x7f8b9000b3e0 comp rx=0 tx=0).stop 2024-05-03T17:54:04.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.632+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/2092469620 shutdown_connections 2024-05-03T17:54:04.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.632+0000 7f8b9ff7e700 1 --2- 172.21.15.122:0/2092469620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b98103f00 0x7f8b981103b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:04.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.632+0000 7f8b9ff7e700 1 --2- 172.21.15.122:0/2092469620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b981095e0 0x7f8b981099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:04.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.632+0000 7f8b9ff7e700 1 --2- 172.21.15.122:0/2092469620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b98103540 0x7f8b981039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:04.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.632+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/2092469620 >> 172.21.15.122:0/2092469620 conn(0x7f8b9807a650 msgr2=0x7f8b9807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:04.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.632+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/2092469620 shutdown_connections 2024-05-03T17:54:04.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.632+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/2092469620 wait complete. 2024-05-03T17:54:04.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.632+0000 7f8b9ff7e700 1 Processor -- start 2024-05-03T17:54:04.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.632+0000 7f8b9ff7e700 1 -- start start 2024-05-03T17:54:04.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9ff7e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b98103540 0x7f8b9819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:04.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9ff7e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b98103f00 0x7f8b9819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:04.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9ff7e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b981095e0 0x7f8b98199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:04.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9ff7e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98113700 con 0x7f8b98103f00 2024-05-03T17:54:04.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9ff7e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98113580 con 0x7f8b981095e0 2024-05-03T17:54:04.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9ff7e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98113880 con 0x7f8b98103540 2024-05-03T17:54:04.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9d519700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b98103f00 0x7f8b9819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:04.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9d519700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b98103f00 0x7f8b9819fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39432/0 (socket says 172.21.15.122:39432) 2024-05-03T17:54:04.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9d519700 1 -- 172.21.15.122:0/293352234 learned_addr learned my addr 172.21.15.122:0/293352234 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:04.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9dd1a700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b98103540 0x7f8b9819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:04.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9e51b700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b981095e0 0x7f8b98199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:04.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9d519700 1 -- 172.21.15.122:0/293352234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b98103540 msgr2=0x7f8b9819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:04.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9d519700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b98103540 0x7f8b9819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:04.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9d519700 1 -- 172.21.15.122:0/293352234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b981095e0 msgr2=0x7f8b98199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:04.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9d519700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b981095e0 0x7f8b98199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:04.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9d519700 1 -- 172.21.15.122:0/293352234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9819a060 con 0x7f8b98103f00 2024-05-03T17:54:04.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9e51b700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b981095e0 0x7f8b98199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:54:04.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9d519700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b98103f00 0x7f8b9819fc80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800d980 tx=0x7f8b8800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:04.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b8effd700 1 -- 172.21.15.122:0/293352234 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88017070 con 0x7f8b98103f00 2024-05-03T17:54:04.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/293352234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9819a350 con 0x7f8b98103f00 2024-05-03T17:54:04.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b8effd700 1 -- 172.21.15.122:0/293352234 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b88005020 con 0x7f8b98103f00 2024-05-03T17:54:04.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b8effd700 1 -- 172.21.15.122:0/293352234 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88009b50 con 0x7f8b98103f00 2024-05-03T17:54:04.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.633+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/293352234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b981a6610 con 0x7f8b98103f00 2024-05-03T17:54:04.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.634+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/293352234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b98063b70 con 0x7f8b98103f00 2024-05-03T17:54:04.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.637+0000 7f8b8effd700 1 -- 172.21.15.122:0/293352234 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8b880051d0 con 0x7f8b98103f00 2024-05-03T17:54:04.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.637+0000 7f8b8effd700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8b840b2910 0x7f8b840b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:04.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.637+0000 7f8b9dd1a700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8b840b2910 0x7f8b840b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:04.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.637+0000 7f8b8effd700 1 -- 172.21.15.122:0/293352234 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f8b880e27a0 con 0x7f8b98103f00 2024-05-03T17:54:04.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.638+0000 7f8b8effd700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137] conn(0x7f8b840bc350 0x7f8b840be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.638+0000 7f8b8effd700 1 -- 172.21.15.122:0/293352234 --> [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b840beec0 con 0x7f8b840bc350 2024-05-03T17:54:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.638+0000 7f8b8effd700 1 -- 172.21.15.122:0/293352234 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f8b880e2c80 con 0x7f8b98103f00 2024-05-03T17:54:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.638+0000 7f8b9dd1a700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8b840b2910 0x7f8b840b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b9819aa40 tx=0x7f8b9400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.638+0000 7f8b9e51b700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137] conn(0x7f8b840bc350 0x7f8b840be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.638+0000 7f8b9e51b700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137] conn(0x7f8b840bc350 0x7f8b840be7f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.639+0000 7f8b8effd700 1 -- 172.21.15.122:0/293352234 <== osd.10 v2:172.21.15.149:6824/1954186137 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b840beec0 con 0x7f8b840bc350 2024-05-03T17:54:04.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.656+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/293352234 --> [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b98063ff0 con 0x7f8b840bc350 2024-05-03T17:54:04.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b8effd700 1 -- 172.21.15.122:0/293352234 <== osd.10 v2:172.21.15.149:6824/1954186137 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8b98063ff0 con 0x7f8b840bc350 2024-05-03T17:54:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/293352234 >> [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137] conn(0x7f8b840bc350 msgr2=0x7f8b840be7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b9ff7e700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137] conn(0x7f8b840bc350 0x7f8b840be7f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/293352234 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8b840b2910 msgr2=0x7f8b840b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b9ff7e700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8b840b2910 0x7f8b840b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b9819aa40 tx=0x7f8b9400a380 comp rx=0 tx=0).stop 2024-05-03T17:54:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/293352234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b98103f00 msgr2=0x7f8b9819fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b9ff7e700 1 --2- 172.21.15.122:0/293352234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b98103f00 0x7f8b9819fc80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800d980 tx=0x7f8b8800dc90 comp rx=0 tx=0).stop 2024-05-03T17:54:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b9e51b700 1 -- 172.21.15.122:0/293352234 reap_dead start 2024-05-03T17:54:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/293352234 shutdown_connections 2024-05-03T17:54:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/293352234 >> 172.21.15.122:0/293352234 conn(0x7f8b9807a650 msgr2=0x7f8b98102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/293352234 shutdown_connections 2024-05-03T17:54:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:04.657+0000 7f8b9ff7e700 1 -- 172.21.15.122:0/293352234 wait complete. 2024-05-03T17:54:04.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:04 smithi149 ceph-mon[33409]: pgmap v331: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:06.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.060+0000 7febf5293700 1 -- 172.21.15.122:0/1969466657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febf0102390 msgr2=0x7febf0102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:06.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.060+0000 7febf5293700 1 --2- 172.21.15.122:0/1969466657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febf0102390 0x7febf0102810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febd8009990 tx=0x7febd8009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:06.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.060+0000 7febf5293700 1 -- 172.21.15.122:0/1969466657 shutdown_connections 2024-05-03T17:54:06.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.060+0000 7febf5293700 1 --2- 172.21.15.122:0/1969466657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf0102d50 0x7febf010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:06.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.060+0000 7febf5293700 1 --2- 172.21.15.122:0/1969466657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf0108400 0x7febf01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:06.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.060+0000 7febf5293700 1 --2- 172.21.15.122:0/1969466657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febf0102390 0x7febf0102810 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:06.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.060+0000 7febf5293700 1 -- 172.21.15.122:0/1969466657 >> 172.21.15.122:0/1969466657 conn(0x7febf0079310 msgr2=0x7febf0079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:06.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.060+0000 7febf5293700 1 -- 172.21.15.122:0/1969466657 shutdown_connections 2024-05-03T17:54:06.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.060+0000 7febf5293700 1 -- 172.21.15.122:0/1969466657 wait complete. 2024-05-03T17:54:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febf5293700 1 Processor -- start 2024-05-03T17:54:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febf5293700 1 -- start start 2024-05-03T17:54:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febf5293700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febf0102390 0x7febf0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febf5293700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf0102d50 0x7febf0198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febf5293700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf0108400 0x7febf0199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febf5293700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7febf0112520 con 0x7febf0102390 2024-05-03T17:54:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febf5293700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7febf01123a0 con 0x7febf0102d50 2024-05-03T17:54:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febf5293700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7febf01126a0 con 0x7febf0108400 2024-05-03T17:54:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febeeffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febf0102390 0x7febf0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febeeffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febf0102390 0x7febf0198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39448/0 (socket says 172.21.15.122:39448) 2024-05-03T17:54:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febeeffd700 1 -- 172.21.15.122:0/4208500524 learned_addr learned my addr 172.21.15.122:0/4208500524 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febe7fff700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf0102d50 0x7febf0198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febef7fe700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf0108400 0x7febf0199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febeeffd700 1 -- 172.21.15.122:0/4208500524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf0108400 msgr2=0x7febf0199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febeeffd700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf0108400 0x7febf0199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febeeffd700 1 -- 172.21.15.122:0/4208500524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf0102d50 msgr2=0x7febf0198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febeeffd700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febf0102d50 0x7febf0198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febeeffd700 1 -- 172.21.15.122:0/4208500524 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf01a8000 con 0x7febf0102390 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febef7fe700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf0108400 0x7febf0199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.061+0000 7febeeffd700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febf0102390 0x7febf0198790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febe000c9b0 tx=0x7febe000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.062+0000 7febecff9700 1 -- 172.21.15.122:0/4208500524 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0007b00 con 0x7febf0102390 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.062+0000 7febecff9700 1 -- 172.21.15.122:0/4208500524 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febe0007ca0 con 0x7febf0102390 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.062+0000 7febf5293700 1 -- 172.21.15.122:0/4208500524 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf01a82f0 con 0x7febf0102390 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.062+0000 7febecff9700 1 -- 172.21.15.122:0/4208500524 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0018730 con 0x7febf0102390 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.062+0000 7febf5293700 1 -- 172.21.15.122:0/4208500524 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf01a8830 con 0x7febf0102390 2024-05-03T17:54:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.062+0000 7febf5293700 1 -- 172.21.15.122:0/4208500524 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7febf0063bb0 con 0x7febf0102390 2024-05-03T17:54:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.064+0000 7febecff9700 1 -- 172.21.15.122:0/4208500524 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7febe000f450 con 0x7febf0102390 2024-05-03T17:54:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.064+0000 7febecff9700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7febdc0b2910 0x7febdc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.065+0000 7febe7fff700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7febdc0b2910 0x7febdc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.065+0000 7febecff9700 1 -- 172.21.15.122:0/4208500524 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7febe00e2750 con 0x7febf0102390 2024-05-03T17:54:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.065+0000 7febecff9700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110] conn(0x7febdc0bc350 0x7febdc0be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.065+0000 7febecff9700 1 -- 172.21.15.122:0/4208500524 --> [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febe001fe50 con 0x7febdc0bc350 2024-05-03T17:54:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.065+0000 7febecff9700 1 -- 172.21.15.122:0/4208500524 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7febe00e2c30 con 0x7febf0102390 2024-05-03T17:54:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.065+0000 7febe7fff700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7febdc0b2910 0x7febdc0b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7febd8006010 tx=0x7febd801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.065+0000 7febef7fe700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110] conn(0x7febdc0bc350 0x7febdc0be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.066+0000 7febef7fe700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110] conn(0x7febdc0bc350 0x7febdc0be7f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:06.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.066+0000 7febecff9700 1 -- 172.21.15.122:0/4208500524 <== osd.2 v2:172.21.15.192:6800/3757191110 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7febe001fe50 con 0x7febdc0bc350 2024-05-03T17:54:06.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.083+0000 7febf5293700 1 -- 172.21.15.122:0/4208500524 --> [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7febf0064030 con 0x7febdc0bc350 2024-05-03T17:54:06.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.083+0000 7febecff9700 1 -- 172.21.15.122:0/4208500524 <== osd.2 v2:172.21.15.192:6800/3757191110 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7febf0064030 con 0x7febdc0bc350 2024-05-03T17:54:06.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.084+0000 7febf5293700 1 -- 172.21.15.122:0/4208500524 >> [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110] conn(0x7febdc0bc350 msgr2=0x7febdc0be7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:06.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.084+0000 7febf5293700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110] conn(0x7febdc0bc350 0x7febdc0be7f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:06.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.084+0000 7febf5293700 1 -- 172.21.15.122:0/4208500524 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7febdc0b2910 msgr2=0x7febdc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:06.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.084+0000 7febf5293700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7febdc0b2910 0x7febdc0b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7febd8006010 tx=0x7febd801a040 comp rx=0 tx=0).stop 2024-05-03T17:54:06.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.084+0000 7febf5293700 1 -- 172.21.15.122:0/4208500524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febf0102390 msgr2=0x7febf0198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:06.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.084+0000 7febf5293700 1 --2- 172.21.15.122:0/4208500524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febf0102390 0x7febf0198790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febe000c9b0 tx=0x7febe000cd70 comp rx=0 tx=0).stop 2024-05-03T17:54:06.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.084+0000 7febef7fe700 1 -- 172.21.15.122:0/4208500524 reap_dead start 2024-05-03T17:54:06.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.084+0000 7febf5293700 1 -- 172.21.15.122:0/4208500524 shutdown_connections 2024-05-03T17:54:06.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.084+0000 7febf5293700 1 -- 172.21.15.122:0/4208500524 >> 172.21.15.122:0/4208500524 conn(0x7febf0079310 msgr2=0x7febf0100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:06.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.084+0000 7febf5293700 1 -- 172.21.15.122:0/4208500524 shutdown_connections 2024-05-03T17:54:06.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.084+0000 7febf5293700 1 -- 172.21.15.122:0/4208500524 wait complete. 2024-05-03T17:54:06.593 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:06 smithi122 ceph-mon[29744]: pgmap v332: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:06.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:06 smithi192 ceph-mon[33753]: pgmap v332: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:06.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:06 smithi149 ceph-mon[33409]: pgmap v332: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.245+0000 7f752631e700 1 -- 172.21.15.122:0/3721728282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7520108410 msgr2=0x7f75201087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.245+0000 7f752631e700 1 --2- 172.21.15.122:0/3721728282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7520108410 0x7f75201087f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7510009990 tx=0x7f7510009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.245+0000 7f752631e700 1 -- 172.21.15.122:0/3721728282 shutdown_connections 2024-05-03T17:54:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.245+0000 7f752631e700 1 --2- 172.21.15.122:0/3721728282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7520102d60 0x7f752010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.245+0000 7f752631e700 1 --2- 172.21.15.122:0/3721728282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7520108410 0x7f75201087f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.245+0000 7f752631e700 1 --2- 172.21.15.122:0/3721728282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75201023a0 0x7f7520102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.245+0000 7f752631e700 1 -- 172.21.15.122:0/3721728282 >> 172.21.15.122:0/3721728282 conn(0x7f7520079350 msgr2=0x7f7520079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.245+0000 7f752631e700 1 -- 172.21.15.122:0/3721728282 shutdown_connections 2024-05-03T17:54:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.245+0000 7f752631e700 1 -- 172.21.15.122:0/3721728282 wait complete. 2024-05-03T17:54:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f752631e700 1 Processor -- start 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f752631e700 1 -- start start 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f752631e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75201023a0 0x7f752019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f752631e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7520102d60 0x7f752019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f752631e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7520108410 0x7f75201999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f752631e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7520112530 con 0x7f75201023a0 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f752631e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f75201123b0 con 0x7f7520102d60 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f752631e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f75201126b0 con 0x7f7520108410 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f751ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75201023a0 0x7f752019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f751ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75201023a0 0x7f752019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39466/0 (socket says 172.21.15.122:39466) 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f751ffff700 1 -- 172.21.15.122:0/663352663 learned_addr learned my addr 172.21.15.122:0/663352663 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f751f7fe700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7520102d60 0x7f752019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.246+0000 7f75248bb700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7520108410 0x7f75201999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.247+0000 7f751ffff700 1 -- 172.21.15.122:0/663352663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7520108410 msgr2=0x7f75201999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.247+0000 7f751ffff700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7520108410 0x7f75201999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.247+0000 7f751ffff700 1 -- 172.21.15.122:0/663352663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7520102d60 msgr2=0x7f752019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.247+0000 7f751ffff700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7520102d60 0x7f752019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.247+0000 7f751ffff700 1 -- 172.21.15.122:0/663352663 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752019a100 con 0x7f75201023a0 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.247+0000 7f751ffff700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75201023a0 0x7f752019f7e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7510000c00 tx=0x7f75100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.247+0000 7f751d7fa700 1 -- 172.21.15.122:0/663352663 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510027070 con 0x7f75201023a0 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.247+0000 7f752631e700 1 -- 172.21.15.122:0/663352663 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752019a390 con 0x7f75201023a0 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.247+0000 7f751d7fa700 1 -- 172.21.15.122:0/663352663 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7510003f20 con 0x7f75201023a0 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.247+0000 7f751d7fa700 1 -- 172.21.15.122:0/663352663 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510022ca0 con 0x7f75201023a0 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.247+0000 7f752631e700 1 -- 172.21.15.122:0/663352663 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75201a66b0 con 0x7f75201023a0 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.248+0000 7f752631e700 1 -- 172.21.15.122:0/663352663 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75201074a0 con 0x7f75201023a0 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.249+0000 7f751d7fa700 1 -- 172.21.15.122:0/663352663 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f751002a070 con 0x7f75201023a0 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.250+0000 7f751d7fa700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f75080b2910 0x7f75080b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.250+0000 7f751f7fe700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f75080b2910 0x7f75080b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.250+0000 7f751d7fa700 1 -- 172.21.15.122:0/663352663 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f75100e3b00 con 0x7f75201023a0 2024-05-03T17:54:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.250+0000 7f751d7fa700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532] conn(0x7f75080bc350 0x7f75080be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.250+0000 7f751d7fa700 1 -- 172.21.15.122:0/663352663 --> [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7510015ec0 con 0x7f75080bc350 2024-05-03T17:54:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.250+0000 7f751d7fa700 1 -- 172.21.15.122:0/663352663 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f75100e3fe0 con 0x7f75201023a0 2024-05-03T17:54:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.250+0000 7f751f7fe700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f75080b2910 0x7f75080b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f75200798e0 tx=0x7f751400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.250+0000 7f75248bb700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532] conn(0x7f75080bc350 0x7f75080be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.251+0000 7f75248bb700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532] conn(0x7f75080bc350 0x7f75080be7f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.252+0000 7f751d7fa700 1 -- 172.21.15.122:0/663352663 <== osd.4 v2:172.21.15.149:6808/3627037532 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7510015ec0 con 0x7f75080bc350 2024-05-03T17:54:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.268+0000 7f752631e700 1 -- 172.21.15.122:0/663352663 --> [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7520103ac0 con 0x7f75080bc350 2024-05-03T17:54:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.269+0000 7f751d7fa700 1 -- 172.21.15.122:0/663352663 <== osd.4 v2:172.21.15.149:6808/3627037532 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7520103ac0 con 0x7f75080bc350 2024-05-03T17:54:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.270+0000 7f752631e700 1 -- 172.21.15.122:0/663352663 >> [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532] conn(0x7f75080bc350 msgr2=0x7f75080be7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.270+0000 7f752631e700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532] conn(0x7f75080bc350 0x7f75080be7f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.270+0000 7f752631e700 1 -- 172.21.15.122:0/663352663 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f75080b2910 msgr2=0x7f75080b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.270+0000 7f752631e700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f75080b2910 0x7f75080b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f75200798e0 tx=0x7f751400b410 comp rx=0 tx=0).stop 2024-05-03T17:54:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.270+0000 7f752631e700 1 -- 172.21.15.122:0/663352663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75201023a0 msgr2=0x7f752019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.270+0000 7f752631e700 1 --2- 172.21.15.122:0/663352663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75201023a0 0x7f752019f7e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7510000c00 tx=0x7f75100047b0 comp rx=0 tx=0).stop 2024-05-03T17:54:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.270+0000 7f75248bb700 1 -- 172.21.15.122:0/663352663 reap_dead start 2024-05-03T17:54:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.270+0000 7f752631e700 1 -- 172.21.15.122:0/663352663 shutdown_connections 2024-05-03T17:54:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.270+0000 7f752631e700 1 -- 172.21.15.122:0/663352663 >> 172.21.15.122:0/663352663 conn(0x7f7520079350 msgr2=0x7f752006b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.270+0000 7f752631e700 1 -- 172.21.15.122:0/663352663 shutdown_connections 2024-05-03T17:54:07.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:06.270+0000 7f752631e700 1 -- 172.21.15.122:0/663352663 wait complete. 2024-05-03T17:54:08.284 INFO:teuthology.orchestra.run.smithi122.stdout:133143986247 2024-05-03T17:54:08.284 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-03T17:54:08.589 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:08 smithi149 ceph-mon[33409]: pgmap v333: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T17:54:08.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:08 smithi192 ceph-mon[33753]: pgmap v333: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T17:54:08.648 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:08 smithi122 ceph-mon[29744]: pgmap v333: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T17:54:08.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.755+0000 7fd6818ce700 1 -- 172.21.15.122:0/2627160271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67c06bb40 msgr2=0x7fd67c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:08.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.755+0000 7fd6818ce700 1 --2- 172.21.15.122:0/2627160271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67c06bb40 0x7fd67c110330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd67400b0a0 tx=0x7fd67400b3b0 comp rx=0 tx=0).stop 2024-05-03T17:54:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.755+0000 7fd6818ce700 1 -- 172.21.15.122:0/2627160271 shutdown_connections 2024-05-03T17:54:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.755+0000 7fd6818ce700 1 --2- 172.21.15.122:0/2627160271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67c06bb40 0x7fd67c110330 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.755+0000 7fd6818ce700 1 --2- 172.21.15.122:0/2627160271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd67c109510 0x7fd67c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.755+0000 7fd6818ce700 1 --2- 172.21.15.122:0/2627160271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd67c06b180 0x7fd67c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.755+0000 7fd6818ce700 1 -- 172.21.15.122:0/2627160271 >> 172.21.15.122:0/2627160271 conn(0x7fd67c079350 msgr2=0x7fd67c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.755+0000 7fd6818ce700 1 -- 172.21.15.122:0/2627160271 shutdown_connections 2024-05-03T17:54:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.755+0000 7fd6818ce700 1 -- 172.21.15.122:0/2627160271 wait complete. 2024-05-03T17:54:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd6818ce700 1 Processor -- start 2024-05-03T17:54:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd6818ce700 1 -- start start 2024-05-03T17:54:08.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd6818ce700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67c06b180 0x7fd67c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:08.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd6818ce700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd67c06bb40 0x7fd67c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:08.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd6818ce700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd67c109510 0x7fd67c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:08.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd6818ce700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c113620 con 0x7fd67c06b180 2024-05-03T17:54:08.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd6818ce700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c1134a0 con 0x7fd67c109510 2024-05-03T17:54:08.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd6818ce700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c1137a0 con 0x7fd67c06bb40 2024-05-03T17:54:08.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd67affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67c06b180 0x7fd67c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:08.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd67affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67c06b180 0x7fd67c19f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49142/0 (socket says 172.21.15.122:49142) 2024-05-03T17:54:08.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd67affd700 1 -- 172.21.15.122:0/2157199981 learned_addr learned my addr 172.21.15.122:0/2157199981 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:08.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd67a7fc700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd67c06bb40 0x7fd67c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:08.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd67b7fe700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd67c109510 0x7fd67c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:08.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd67affd700 1 -- 172.21.15.122:0/2157199981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd67c06bb40 msgr2=0x7fd67c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:08.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd67affd700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd67c06bb40 0x7fd67c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:08.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd67affd700 1 -- 172.21.15.122:0/2157199981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd67c109510 msgr2=0x7fd67c199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:08.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd67affd700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd67c109510 0x7fd67c199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:08.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.756+0000 7fd67affd700 1 -- 172.21.15.122:0/2157199981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c19a0e0 con 0x7fd67c06b180 2024-05-03T17:54:08.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.757+0000 7fd67affd700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67c06b180 0x7fd67c19f7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00d950 tx=0x7fd66c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:08.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.757+0000 7fd6808cc700 1 -- 172.21.15.122:0/2157199981 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c016070 con 0x7fd67c06b180 2024-05-03T17:54:08.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.757+0000 7fd6808cc700 1 -- 172.21.15.122:0/2157199981 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd66c010460 con 0x7fd67c06b180 2024-05-03T17:54:08.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.757+0000 7fd6818ce700 1 -- 172.21.15.122:0/2157199981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c19a3d0 con 0x7fd67c06b180 2024-05-03T17:54:08.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.757+0000 7fd6808cc700 1 -- 172.21.15.122:0/2157199981 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c00f620 con 0x7fd67c06b180 2024-05-03T17:54:08.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.757+0000 7fd6818ce700 1 -- 172.21.15.122:0/2157199981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c1a6690 con 0x7fd67c06b180 2024-05-03T17:54:08.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.757+0000 7fd6818ce700 1 -- 172.21.15.122:0/2157199981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd67c063bb0 con 0x7fd67c06b180 2024-05-03T17:54:08.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.759+0000 7fd6808cc700 1 -- 172.21.15.122:0/2157199981 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fd66c019070 con 0x7fd67c06b180 2024-05-03T17:54:08.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.759+0000 7fd6808cc700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd6640b2910 0x7fd6640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:08.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.760+0000 7fd67a7fc700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd6640b2910 0x7fd6640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:08.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.760+0000 7fd6808cc700 1 -- 172.21.15.122:0/2157199981 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fd66c0e23c0 con 0x7fd67c06b180 2024-05-03T17:54:08.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.760+0000 7fd6808cc700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764] conn(0x7fd6640bc350 0x7fd6640be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:08.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.760+0000 7fd6808cc700 1 -- 172.21.15.122:0/2157199981 --> [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6640beec0 con 0x7fd6640bc350 2024-05-03T17:54:08.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.760+0000 7fd6808cc700 1 -- 172.21.15.122:0/2157199981 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fd66c0e28a0 con 0x7fd67c06b180 2024-05-03T17:54:08.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.760+0000 7fd67a7fc700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd6640b2910 0x7fd6640b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd670005fd0 tx=0x7fd670005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:08.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.760+0000 7fd67b7fe700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764] conn(0x7fd6640bc350 0x7fd6640be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:08.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.760+0000 7fd67b7fe700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764] conn(0x7fd6640bc350 0x7fd6640be7f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:08.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.761+0000 7fd6808cc700 1 -- 172.21.15.122:0/2157199981 <== osd.3 v2:172.21.15.122:6810/2359143764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6640beec0 con 0x7fd6640bc350 2024-05-03T17:54:08.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.777+0000 7fd6818ce700 1 -- 172.21.15.122:0/2157199981 --> [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd67c063fb0 con 0x7fd6640bc350 2024-05-03T17:54:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd6808cc700 1 -- 172.21.15.122:0/2157199981 <== osd.3 v2:172.21.15.122:6810/2359143764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd67c063fb0 con 0x7fd6640bc350 2024-05-03T17:54:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd6818ce700 1 -- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764] conn(0x7fd6640bc350 msgr2=0x7fd6640be7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd6818ce700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764] conn(0x7fd6640bc350 0x7fd6640be7f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd6818ce700 1 -- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd6640b2910 msgr2=0x7fd6640b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd6818ce700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd6640b2910 0x7fd6640b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd670005fd0 tx=0x7fd670005f60 comp rx=0 tx=0).stop 2024-05-03T17:54:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd6818ce700 1 -- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67c06b180 msgr2=0x7fd67c19f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd6818ce700 1 --2- 172.21.15.122:0/2157199981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67c06b180 0x7fd67c19f7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00d950 tx=0x7fd66c00dd10 comp rx=0 tx=0).stop 2024-05-03T17:54:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd67b7fe700 1 -- 172.21.15.122:0/2157199981 reap_dead start 2024-05-03T17:54:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd6818ce700 1 -- 172.21.15.122:0/2157199981 shutdown_connections 2024-05-03T17:54:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd6818ce700 1 -- 172.21.15.122:0/2157199981 >> 172.21.15.122:0/2157199981 conn(0x7fd67c079350 msgr2=0x7fd67c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd6818ce700 1 -- 172.21.15.122:0/2157199981 shutdown_connections 2024-05-03T17:54:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:08.778+0000 7fd6818ce700 1 -- 172.21.15.122:0/2157199981 wait complete. 2024-05-03T17:54:09.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.234+0000 7fdfee8d5700 1 -- 172.21.15.122:0/2253871597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfe8102d60 msgr2=0x7fdfe810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.234+0000 7fdfee8d5700 1 --2- 172.21.15.122:0/2253871597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfe8102d60 0x7fdfe810f250 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000b0a0 tx=0x7fdfe000b3b0 comp rx=0 tx=0).stop 2024-05-03T17:54:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.234+0000 7fdfee8d5700 1 -- 172.21.15.122:0/2253871597 shutdown_connections 2024-05-03T17:54:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.234+0000 7fdfee8d5700 1 --2- 172.21.15.122:0/2253871597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfe8102d60 0x7fdfe810f250 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.234+0000 7fdfee8d5700 1 --2- 172.21.15.122:0/2253871597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe8108410 0x7fdfe81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.234+0000 7fdfee8d5700 1 --2- 172.21.15.122:0/2253871597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe81023a0 0x7fdfe8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.234+0000 7fdfee8d5700 1 -- 172.21.15.122:0/2253871597 >> 172.21.15.122:0/2253871597 conn(0x7fdfe8079350 msgr2=0x7fdfe8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.234+0000 7fdfee8d5700 1 -- 172.21.15.122:0/2253871597 shutdown_connections 2024-05-03T17:54:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.234+0000 7fdfee8d5700 1 -- 172.21.15.122:0/2253871597 wait complete. 2024-05-03T17:54:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfee8d5700 1 Processor -- start 2024-05-03T17:54:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfee8d5700 1 -- start start 2024-05-03T17:54:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfee8d5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe81023a0 0x7fdfe81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfee8d5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe8102d60 0x7fdfe8198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfee8d5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfe8108410 0x7fdfe8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfee8d5700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8112550 con 0x7fdfe8108410 2024-05-03T17:54:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfee8d5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81123d0 con 0x7fdfe81023a0 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfee8d5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe81126d0 con 0x7fdfe8102d60 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfece72700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfe8108410 0x7fdfe8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfece72700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfe8108410 0x7fdfe8199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49156/0 (socket says 172.21.15.122:49156) 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfece72700 1 -- 172.21.15.122:0/1988055633 learned_addr learned my addr 172.21.15.122:0/1988055633 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfe7fff700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe81023a0 0x7fdfe81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.235+0000 7fdfe77fe700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe8102d60 0x7fdfe8198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfece72700 1 -- 172.21.15.122:0/1988055633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe8102d60 msgr2=0x7fdfe8198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfece72700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe8102d60 0x7fdfe8198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfece72700 1 -- 172.21.15.122:0/1988055633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe81023a0 msgr2=0x7fdfe81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfece72700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe81023a0 0x7fdfe81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfece72700 1 -- 172.21.15.122:0/1988055633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe81a7fe0 con 0x7fdfe8108410 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfe77fe700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe8102d60 0x7fdfe8198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:54:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfe7fff700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe81023a0 0x7fdfe81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T17:54:09.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfece72700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfe8108410 0x7fdfe8199160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000b780 tx=0x7fdfe0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:09.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfe57fa700 1 -- 172.21.15.122:0/1988055633 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0021070 con 0x7fdfe8108410 2024-05-03T17:54:09.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfee8d5700 1 -- 172.21.15.122:0/1988055633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe81a8270 con 0x7fdfe8108410 2024-05-03T17:54:09.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfe57fa700 1 -- 172.21.15.122:0/1988055633 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfe0007e30 con 0x7fdfe8108410 2024-05-03T17:54:09.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfe57fa700 1 -- 172.21.15.122:0/1988055633 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00047b0 con 0x7fdfe8108410 2024-05-03T17:54:09.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.236+0000 7fdfee8d5700 1 -- 172.21.15.122:0/1988055633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe81a8720 con 0x7fdfe8108410 2024-05-03T17:54:09.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.237+0000 7fdfee8d5700 1 -- 172.21.15.122:0/1988055633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdfe8063c30 con 0x7fdfe8108410 2024-05-03T17:54:09.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.239+0000 7fdfe57fa700 1 -- 172.21.15.122:0/1988055633 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fdfe0007980 con 0x7fdfe8108410 2024-05-03T17:54:09.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.239+0000 7fdfe57fa700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fdfd00b2910 0x7fdfd00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:09.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.240+0000 7fdfe7fff700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fdfd00b2910 0x7fdfd00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:09.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.240+0000 7fdfe57fa700 1 -- 172.21.15.122:0/1988055633 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fdfe00e35a0 con 0x7fdfe8108410 2024-05-03T17:54:09.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.240+0000 7fdfe57fa700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421] conn(0x7fdfd00bc350 0x7fdfd00be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:09.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.240+0000 7fdfe57fa700 1 -- 172.21.15.122:0/1988055633 --> [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdfe001ce50 con 0x7fdfd00bc350 2024-05-03T17:54:09.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.240+0000 7fdfe57fa700 1 -- 172.21.15.122:0/1988055633 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fdfe00e3a80 con 0x7fdfe8108410 2024-05-03T17:54:09.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.240+0000 7fdfe77fe700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421] conn(0x7fdfd00bc350 0x7fdfd00be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:09.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.240+0000 7fdfe7fff700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fdfd00b2910 0x7fdfd00b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009fd0 tx=0x7fdfd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:09.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.240+0000 7fdfe77fe700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421] conn(0x7fdfd00bc350 0x7fdfd00be7f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:09.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.241+0000 7fdfe57fa700 1 -- 172.21.15.122:0/1988055633 <== osd.9 v2:172.21.15.122:6826/1773691421 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdfe001ce50 con 0x7fdfd00bc350 2024-05-03T17:54:09.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.258+0000 7fdfee8d5700 1 -- 172.21.15.122:0/1988055633 --> [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdfe80640b0 con 0x7fdfd00bc350 2024-05-03T17:54:09.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.258+0000 7fdfe57fa700 1 -- 172.21.15.122:0/1988055633 <== osd.9 v2:172.21.15.122:6826/1773691421 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdfe80640b0 con 0x7fdfd00bc350 2024-05-03T17:54:09.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.258+0000 7fdfee8d5700 1 -- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421] conn(0x7fdfd00bc350 msgr2=0x7fdfd00be7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:09.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.258+0000 7fdfee8d5700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421] conn(0x7fdfd00bc350 0x7fdfd00be7f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:09.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.258+0000 7fdfee8d5700 1 -- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fdfd00b2910 msgr2=0x7fdfd00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:09.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.258+0000 7fdfee8d5700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fdfd00b2910 0x7fdfd00b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009fd0 tx=0x7fdfd8009260 comp rx=0 tx=0).stop 2024-05-03T17:54:09.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.258+0000 7fdfee8d5700 1 -- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfe8108410 msgr2=0x7fdfe8199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:09.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.258+0000 7fdfee8d5700 1 --2- 172.21.15.122:0/1988055633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfe8108410 0x7fdfe8199160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000b780 tx=0x7fdfe0004d00 comp rx=0 tx=0).stop 2024-05-03T17:54:09.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.258+0000 7fdfece72700 1 -- 172.21.15.122:0/1988055633 reap_dead start 2024-05-03T17:54:09.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.259+0000 7fdfee8d5700 1 -- 172.21.15.122:0/1988055633 shutdown_connections 2024-05-03T17:54:09.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.259+0000 7fdfee8d5700 1 -- 172.21.15.122:0/1988055633 >> 172.21.15.122:0/1988055633 conn(0x7fdfe8079350 msgr2=0x7fdfe8100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.259+0000 7fdfee8d5700 1 -- 172.21.15.122:0/1988055633 shutdown_connections 2024-05-03T17:54:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:09.259+0000 7fdfee8d5700 1 -- 172.21.15.122:0/1988055633 wait complete. 2024-05-03T17:54:10.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.314+0000 7fcc0a05d700 1 -- 172.21.15.122:0/619614340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04108460 msgr2=0x7fcc04108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:10.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.314+0000 7fcc0a05d700 1 --2- 172.21.15.122:0/619614340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04108460 0x7fcc04108840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009990 tx=0x7fcbf4009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:10.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.314+0000 7fcc0a05d700 1 -- 172.21.15.122:0/619614340 shutdown_connections 2024-05-03T17:54:10.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.314+0000 7fcc0a05d700 1 --2- 172.21.15.122:0/619614340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc04103ef0 0x7fcc041103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:10.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.314+0000 7fcc0a05d700 1 --2- 172.21.15.122:0/619614340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04108460 0x7fcc04108840 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:10.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.314+0000 7fcc0a05d700 1 --2- 172.21.15.122:0/619614340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc04103550 0x7fcc041039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:10.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.314+0000 7fcc0a05d700 1 -- 172.21.15.122:0/619614340 >> 172.21.15.122:0/619614340 conn(0x7fcc0407a600 msgr2=0x7fcc0407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:10.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.314+0000 7fcc0a05d700 1 -- 172.21.15.122:0/619614340 shutdown_connections 2024-05-03T17:54:10.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.314+0000 7fcc0a05d700 1 -- 172.21.15.122:0/619614340 wait complete. 2024-05-03T17:54:10.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.314+0000 7fcc0a05d700 1 Processor -- start 2024-05-03T17:54:10.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.314+0000 7fcc0a05d700 1 -- start start 2024-05-03T17:54:10.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc0a05d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04103550 0x7fcc0419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:10.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc0a05d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc04103ef0 0x7fcc0419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc0a05d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc04108460 0x7fcc041999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc0a05d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04113700 con 0x7fcc04103550 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc0a05d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04113580 con 0x7fcc04103ef0 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc0a05d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04113880 con 0x7fcc04108460 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc037fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04103550 0x7fcc0419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc037fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04103550 0x7fcc0419f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49176/0 (socket says 172.21.15.122:49176) 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc037fe700 1 -- 172.21.15.122:0/4265248527 learned_addr learned my addr 172.21.15.122:0/4265248527 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcbfaffd700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc04103ef0 0x7fcc0419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc03fff700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc04108460 0x7fcc041999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc037fe700 1 -- 172.21.15.122:0/4265248527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc04108460 msgr2=0x7fcc041999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc037fe700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc04108460 0x7fcc041999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc037fe700 1 -- 172.21.15.122:0/4265248527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc04103ef0 msgr2=0x7fcc0419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc037fe700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc04103ef0 0x7fcc0419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc037fe700 1 -- 172.21.15.122:0/4265248527 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0419a100 con 0x7fcc04103550 2024-05-03T17:54:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc037fe700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04103550 0x7fcc0419f790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4000c00 tx=0x7fcbf4009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc017fa700 1 -- 172.21.15.122:0/4265248527 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4026070 con 0x7fcc04103550 2024-05-03T17:54:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc017fa700 1 -- 172.21.15.122:0/4265248527 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf40040e0 con 0x7fcc04103550 2024-05-03T17:54:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc0a05d700 1 -- 172.21.15.122:0/4265248527 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0419a390 con 0x7fcc04103550 2024-05-03T17:54:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.315+0000 7fcc017fa700 1 -- 172.21.15.122:0/4265248527 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf400fd30 con 0x7fcc04103550 2024-05-03T17:54:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.316+0000 7fcc0a05d700 1 -- 172.21.15.122:0/4265248527 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc040756d0 con 0x7fcc04103550 2024-05-03T17:54:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.316+0000 7fcc0a05d700 1 -- 172.21.15.122:0/4265248527 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc04063c30 con 0x7fcc04103550 2024-05-03T17:54:10.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.318+0000 7fcc017fa700 1 -- 172.21.15.122:0/4265248527 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fcbf4029070 con 0x7fcc04103550 2024-05-03T17:54:10.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.318+0000 7fcc017fa700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcbf00b2990 0x7fcbf00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:10.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.318+0000 7fcbfaffd700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcbf00b2990 0x7fcbf00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:10.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.319+0000 7fcc017fa700 1 -- 172.21.15.122:0/4265248527 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fcbf40e3800 con 0x7fcc04103550 2024-05-03T17:54:10.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.319+0000 7fcc017fa700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610] conn(0x7fcbf00bc3d0 0x7fcbf00be870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:10.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.319+0000 7fcc017fa700 1 -- 172.21.15.122:0/4265248527 --> [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcbf00bef40 con 0x7fcbf00bc3d0 2024-05-03T17:54:10.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.319+0000 7fcc017fa700 1 -- 172.21.15.122:0/4265248527 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fcbf40e3ce0 con 0x7fcc04103550 2024-05-03T17:54:10.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.319+0000 7fcc03fff700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610] conn(0x7fcbf00bc3d0 0x7fcbf00be870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:10.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.319+0000 7fcc03fff700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610] conn(0x7fcbf00bc3d0 0x7fcbf00be870 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:10.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.320+0000 7fcbfaffd700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcbf00b2990 0x7fcbf00b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcc0407ab70 tx=0x7fcbec00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:10.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.320+0000 7fcc017fa700 1 -- 172.21.15.122:0/4265248527 <== osd.0 v2:172.21.15.122:6802/1329364610 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbf00bef40 con 0x7fcbf00bc3d0 2024-05-03T17:54:10.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc0a05d700 1 -- 172.21.15.122:0/4265248527 --> [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc04064030 con 0x7fcbf00bc3d0 2024-05-03T17:54:10.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc017fa700 1 -- 172.21.15.122:0/4265248527 <== osd.0 v2:172.21.15.122:6802/1329364610 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcc04064030 con 0x7fcbf00bc3d0 2024-05-03T17:54:10.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc0a05d700 1 -- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610] conn(0x7fcbf00bc3d0 msgr2=0x7fcbf00be870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:10.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc0a05d700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610] conn(0x7fcbf00bc3d0 0x7fcbf00be870 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:10.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc0a05d700 1 -- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcbf00b2990 msgr2=0x7fcbf00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:10.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc0a05d700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fcbf00b2990 0x7fcbf00b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcc0407ab70 tx=0x7fcbec00a380 comp rx=0 tx=0).stop 2024-05-03T17:54:10.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc0a05d700 1 -- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04103550 msgr2=0x7fcc0419f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:10.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc0a05d700 1 --2- 172.21.15.122:0/4265248527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04103550 0x7fcc0419f790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4000c00 tx=0x7fcbf4009f50 comp rx=0 tx=0).stop 2024-05-03T17:54:10.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc03fff700 1 -- 172.21.15.122:0/4265248527 reap_dead start 2024-05-03T17:54:10.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc0a05d700 1 -- 172.21.15.122:0/4265248527 shutdown_connections 2024-05-03T17:54:10.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc0a05d700 1 -- 172.21.15.122:0/4265248527 >> 172.21.15.122:0/4265248527 conn(0x7fcc0407a600 msgr2=0x7fcc04101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:10.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc0a05d700 1 -- 172.21.15.122:0/4265248527 shutdown_connections 2024-05-03T17:54:10.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:10.337+0000 7fcc0a05d700 1 -- 172.21.15.122:0/4265248527 wait complete. 2024-05-03T17:54:10.342 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:10 smithi122 ceph-mon[29744]: pgmap v334: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:10.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:10 smithi192 ceph-mon[33753]: pgmap v334: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:10.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:10 smithi149 ceph-mon[33409]: pgmap v334: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:11.131 INFO:teuthology.orchestra.run.smithi122.stdout:107374182476 2024-05-03T17:54:11.131 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-03T17:54:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.782+0000 7f5de6aa2700 1 -- 172.21.15.122:0/669977308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5de01037b0 msgr2=0x7f5de0103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.782+0000 7f5de6aa2700 1 --2- 172.21.15.122:0/669977308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5de01037b0 0x7f5de0103b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40099c0 tx=0x7f5dd4009cd0 comp rx=0 tx=0).stop 2024-05-03T17:54:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.782+0000 7f5de6aa2700 1 -- 172.21.15.122:0/669977308 shutdown_connections 2024-05-03T17:54:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.782+0000 7f5de6aa2700 1 --2- 172.21.15.122:0/669977308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5de0108880 0x7f5de0110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.782+0000 7f5de6aa2700 1 --2- 172.21.15.122:0/669977308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5de01037b0 0x7f5de0103b90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.782+0000 7f5de6aa2700 1 --2- 172.21.15.122:0/669977308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5de0104160 0x7f5de0108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.782+0000 7f5de6aa2700 1 -- 172.21.15.122:0/669977308 >> 172.21.15.122:0/669977308 conn(0x7f5de007a580 msgr2=0x7f5de007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.782+0000 7f5de6aa2700 1 -- 172.21.15.122:0/669977308 shutdown_connections 2024-05-03T17:54:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.782+0000 7f5de6aa2700 1 -- 172.21.15.122:0/669977308 wait complete. 2024-05-03T17:54:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.782+0000 7f5de6aa2700 1 Processor -- start 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de6aa2700 1 -- start start 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de6aa2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5de01037b0 0x7f5de0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de6aa2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5de0104160 0x7f5de019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de6aa2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5de0108880 0x7f5de019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de6aa2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0113650 con 0x7f5de0108880 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de6aa2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01134d0 con 0x7f5de0104160 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de6aa2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01137d0 con 0x7f5de01037b0 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de483e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5de01037b0 0x7f5de0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de483e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5de01037b0 0x7f5de0199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.122:60556/0 (socket says 172.21.15.122:60556) 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de483e700 1 -- 172.21.15.122:0/827777129 learned_addr learned my addr 172.21.15.122:0/827777129 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de503f700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5de0108880 0x7f5de019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5ddffff700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5de0104160 0x7f5de019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de483e700 1 -- 172.21.15.122:0/827777129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5de0104160 msgr2=0x7f5de019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de483e700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5de0104160 0x7f5de019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de483e700 1 -- 172.21.15.122:0/827777129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5de0108880 msgr2=0x7f5de019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de483e700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5de0108880 0x7f5de019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.783+0000 7f5de483e700 1 -- 172.21.15.122:0/827777129 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de019eae0 con 0x7f5de01037b0 2024-05-03T17:54:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.784+0000 7f5de483e700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5de01037b0 0x7f5de0199ae0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400b580 tx=0x7f5dd4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.784+0000 7f5dddffb700 1 -- 172.21.15.122:0/827777129 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd402b070 con 0x7f5de01037b0 2024-05-03T17:54:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.784+0000 7f5dddffb700 1 -- 172.21.15.122:0/827777129 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd40041a0 con 0x7f5de01037b0 2024-05-03T17:54:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.784+0000 7f5dddffb700 1 -- 172.21.15.122:0/827777129 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd401d450 con 0x7f5de01037b0 2024-05-03T17:54:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.784+0000 7f5de6aa2700 1 -- 172.21.15.122:0/827777129 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de019ed70 con 0x7f5de01037b0 2024-05-03T17:54:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.784+0000 7f5de6aa2700 1 -- 172.21.15.122:0/827777129 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de019f200 con 0x7f5de01037b0 2024-05-03T17:54:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.785+0000 7f5de6aa2700 1 -- 172.21.15.122:0/827777129 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5de0063c30 con 0x7f5de01037b0 2024-05-03T17:54:11.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.787+0000 7f5dddffb700 1 -- 172.21.15.122:0/827777129 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f5dd402e070 con 0x7f5de01037b0 2024-05-03T17:54:11.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.787+0000 7f5dddffb700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5dd00b2910 0x7f5dd00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:11.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.788+0000 7f5ddffff700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5dd00b2910 0x7f5dd00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:11.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.788+0000 7f5dddffb700 1 -- 172.21.15.122:0/827777129 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f5dd40e75e0 con 0x7f5de01037b0 2024-05-03T17:54:11.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.788+0000 7f5dddffb700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302] conn(0x7f5dd00bc350 0x7f5dd00be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:11.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.788+0000 7f5dddffb700 1 -- 172.21.15.122:0/827777129 --> [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5dd00beec0 con 0x7f5dd00bc350 2024-05-03T17:54:11.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.788+0000 7f5dddffb700 1 -- 172.21.15.122:0/827777129 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f5dd40e7990 con 0x7f5de01037b0 2024-05-03T17:54:11.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.788+0000 7f5ddffff700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5dd00b2910 0x7f5dd00b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc005950 tx=0x7f5dcc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:11.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.788+0000 7f5de503f700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302] conn(0x7f5dd00bc350 0x7f5dd00be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:11.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.788+0000 7f5de503f700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302] conn(0x7f5dd00bc350 0x7f5dd00be7f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:11.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.789+0000 7f5dddffb700 1 -- 172.21.15.122:0/827777129 <== osd.1 v2:172.21.15.149:6800/1402463302 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5dd00beec0 con 0x7f5dd00bc350 2024-05-03T17:54:11.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.806+0000 7f5de6aa2700 1 -- 172.21.15.122:0/827777129 --> [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5de00640b0 con 0x7f5dd00bc350 2024-05-03T17:54:11.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5dddffb700 1 -- 172.21.15.122:0/827777129 <== osd.1 v2:172.21.15.149:6800/1402463302 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5de00640b0 con 0x7f5dd00bc350 2024-05-03T17:54:11.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5de6aa2700 1 -- 172.21.15.122:0/827777129 >> [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302] conn(0x7f5dd00bc350 msgr2=0x7f5dd00be7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:11.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5de6aa2700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302] conn(0x7f5dd00bc350 0x7f5dd00be7f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:11.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5de6aa2700 1 -- 172.21.15.122:0/827777129 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5dd00b2910 msgr2=0x7f5dd00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:11.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5de6aa2700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f5dd00b2910 0x7f5dd00b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc005950 tx=0x7f5dcc0058e0 comp rx=0 tx=0).stop 2024-05-03T17:54:11.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5de6aa2700 1 -- 172.21.15.122:0/827777129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5de01037b0 msgr2=0x7f5de0199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:11.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5de6aa2700 1 --2- 172.21.15.122:0/827777129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5de01037b0 0x7f5de0199ae0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400b580 tx=0x7f5dd4005e80 comp rx=0 tx=0).stop 2024-05-03T17:54:11.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5de503f700 1 -- 172.21.15.122:0/827777129 reap_dead start 2024-05-03T17:54:11.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5de6aa2700 1 -- 172.21.15.122:0/827777129 shutdown_connections 2024-05-03T17:54:11.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5de6aa2700 1 -- 172.21.15.122:0/827777129 >> 172.21.15.122:0/827777129 conn(0x7f5de007a580 msgr2=0x7f5de0102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:11.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5de6aa2700 1 -- 172.21.15.122:0/827777129 shutdown_connections 2024-05-03T17:54:11.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:11.807+0000 7f5de6aa2700 1 -- 172.21.15.122:0/827777129 wait complete. 2024-05-03T17:54:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:253403070510 2024-05-03T17:54:12.226 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-03T17:54:12.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.421+0000 7f7a45a58700 1 -- 172.21.15.122:0/4170868234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a40104180 msgr2=0x7f7a401103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:12.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.421+0000 7f7a45a58700 1 --2- 172.21.15.122:0/4170868234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a40104180 0x7f7a401103f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800b0a0 tx=0x7f7a3800b3b0 comp rx=0 tx=0).stop 2024-05-03T17:54:12.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.421+0000 7f7a45a58700 1 -- 172.21.15.122:0/4170868234 shutdown_connections 2024-05-03T17:54:12.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.421+0000 7f7a45a58700 1 --2- 172.21.15.122:0/4170868234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a40104180 0x7f7a401103f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:12.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.421+0000 7f7a45a58700 1 --2- 172.21.15.122:0/4170868234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a40109660 0x7f7a40109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:12.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.421+0000 7f7a45a58700 1 --2- 172.21.15.122:0/4170868234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a401037c0 0x7f7a40103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:12.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.421+0000 7f7a45a58700 1 -- 172.21.15.122:0/4170868234 >> 172.21.15.122:0/4170868234 conn(0x7f7a4007b430 msgr2=0x7f7a4007b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:12.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.421+0000 7f7a45a58700 1 -- 172.21.15.122:0/4170868234 shutdown_connections 2024-05-03T17:54:12.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.421+0000 7f7a45a58700 1 -- 172.21.15.122:0/4170868234 wait complete. 2024-05-03T17:54:12.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a45a58700 1 Processor -- start 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a45a58700 1 -- start start 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a45a58700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a401037c0 0x7f7a4019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a45a58700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a40104180 0x7f7a4019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a45a58700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a40109660 0x7f7a401999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a45a58700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a40113780 con 0x7f7a401037c0 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a45a58700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a40113600 con 0x7f7a40109660 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a45a58700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7a40113900 con 0x7f7a40104180 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a3effd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a401037c0 0x7f7a4019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a3effd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a401037c0 0x7f7a4019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49222/0 (socket says 172.21.15.122:49222) 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a3effd700 1 -- 172.21.15.122:0/807915715 learned_addr learned my addr 172.21.15.122:0/807915715 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a3f7fe700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a40109660 0x7f7a401999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.422+0000 7f7a37fff700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a40104180 0x7f7a4019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:12.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.423+0000 7f7a3effd700 1 -- 172.21.15.122:0/807915715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a40104180 msgr2=0x7f7a4019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:12.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.423+0000 7f7a3effd700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a40104180 0x7f7a4019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:12.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.423+0000 7f7a3effd700 1 -- 172.21.15.122:0/807915715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a40109660 msgr2=0x7f7a401999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:12.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.423+0000 7f7a3effd700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a40109660 0x7f7a401999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:12.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.423+0000 7f7a3effd700 1 -- 172.21.15.122:0/807915715 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4019a100 con 0x7f7a401037c0 2024-05-03T17:54:12.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.423+0000 7f7a3effd700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a401037c0 0x7f7a4019f790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000e9c0 tx=0x7f7a3000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:12.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.423+0000 7f7a44a56700 1 -- 172.21.15.122:0/807915715 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000cbd0 con 0x7f7a401037c0 2024-05-03T17:54:12.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.423+0000 7f7a44a56700 1 -- 172.21.15.122:0/807915715 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a3000cd70 con 0x7f7a401037c0 2024-05-03T17:54:12.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.423+0000 7f7a45a58700 1 -- 172.21.15.122:0/807915715 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4019a3f0 con 0x7f7a401037c0 2024-05-03T17:54:12.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.424+0000 7f7a44a56700 1 -- 172.21.15.122:0/807915715 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30010430 con 0x7f7a401037c0 2024-05-03T17:54:12.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.424+0000 7f7a45a58700 1 -- 172.21.15.122:0/807915715 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a401a6660 con 0x7f7a401037c0 2024-05-03T17:54:12.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.424+0000 7f7a45a58700 1 -- 172.21.15.122:0/807915715 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a40063bb0 con 0x7f7a401037c0 2024-05-03T17:54:12.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.425+0000 7f7a44a56700 1 -- 172.21.15.122:0/807915715 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f7a30004750 con 0x7f7a401037c0 2024-05-03T17:54:12.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.426+0000 7f7a44a56700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f7a2c0b2990 0x7f7a2c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:12.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.426+0000 7f7a37fff700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f7a2c0b2990 0x7f7a2c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:12.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.426+0000 7f7a44a56700 1 -- 172.21.15.122:0/807915715 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f7a300e2600 con 0x7f7a401037c0 2024-05-03T17:54:12.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.426+0000 7f7a44a56700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744] conn(0x7f7a2c0bc3d0 0x7f7a2c0be870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:12.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.426+0000 7f7a44a56700 1 -- 172.21.15.122:0/807915715 --> [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a3001ee50 con 0x7f7a2c0bc3d0 2024-05-03T17:54:12.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.426+0000 7f7a44a56700 1 -- 172.21.15.122:0/807915715 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f7a300e9050 con 0x7f7a401037c0 2024-05-03T17:54:12.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.426+0000 7f7a37fff700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f7a2c0b2990 0x7f7a2c0b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7a28005950 tx=0x7f7a280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:12.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.427+0000 7f7a3f7fe700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744] conn(0x7f7a2c0bc3d0 0x7f7a2c0be870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:12.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.427+0000 7f7a3f7fe700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744] conn(0x7f7a2c0bc3d0 0x7f7a2c0be870 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:12.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.428+0000 7f7a44a56700 1 -- 172.21.15.122:0/807915715 <== osd.8 v2:172.21.15.192:6816/2080816744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a3001ee50 con 0x7f7a2c0bc3d0 2024-05-03T17:54:12.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.446+0000 7f7a45a58700 1 -- 172.21.15.122:0/807915715 --> [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a40064030 con 0x7f7a2c0bc3d0 2024-05-03T17:54:12.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.446+0000 7f7a44a56700 1 -- 172.21.15.122:0/807915715 <== osd.8 v2:172.21.15.192:6816/2080816744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7a40064030 con 0x7f7a2c0bc3d0 2024-05-03T17:54:12.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.446+0000 7f7a45a58700 1 -- 172.21.15.122:0/807915715 >> [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744] conn(0x7f7a2c0bc3d0 msgr2=0x7f7a2c0be870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:12.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.446+0000 7f7a45a58700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744] conn(0x7f7a2c0bc3d0 0x7f7a2c0be870 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:12.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.447+0000 7f7a45a58700 1 -- 172.21.15.122:0/807915715 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f7a2c0b2990 msgr2=0x7f7a2c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:12.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.447+0000 7f7a45a58700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f7a2c0b2990 0x7f7a2c0b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7a28005950 tx=0x7f7a280058e0 comp rx=0 tx=0).stop 2024-05-03T17:54:12.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.447+0000 7f7a45a58700 1 -- 172.21.15.122:0/807915715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a401037c0 msgr2=0x7f7a4019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:12.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.447+0000 7f7a45a58700 1 --2- 172.21.15.122:0/807915715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a401037c0 0x7f7a4019f790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000e9c0 tx=0x7f7a3000ed80 comp rx=0 tx=0).stop 2024-05-03T17:54:12.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.447+0000 7f7a3f7fe700 1 -- 172.21.15.122:0/807915715 reap_dead start 2024-05-03T17:54:12.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.447+0000 7f7a45a58700 1 -- 172.21.15.122:0/807915715 shutdown_connections 2024-05-03T17:54:12.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.447+0000 7f7a45a58700 1 -- 172.21.15.122:0/807915715 >> 172.21.15.122:0/807915715 conn(0x7f7a4007b430 msgr2=0x7f7a401022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:12.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.447+0000 7f7a45a58700 1 -- 172.21.15.122:0/807915715 shutdown_connections 2024-05-03T17:54:12.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:12.447+0000 7f7a45a58700 1 -- 172.21.15.122:0/807915715 wait complete. 2024-05-03T17:54:12.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:12 smithi192 ceph-mon[33753]: pgmap v335: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:12.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:12 smithi122 ceph-mon[29744]: pgmap v335: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:12.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:12 smithi149 ceph-mon[33409]: pgmap v335: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.008+0000 7fc2847fb700 1 -- 172.21.15.122:0/1558238090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc27c06ba10 msgr2=0x7fc27c06be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.008+0000 7fc2847fb700 1 --2- 172.21.15.122:0/1558238090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc27c06ba10 0x7fc27c06be90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009990 tx=0x7fc26c009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.008+0000 7fc2847fb700 1 -- 172.21.15.122:0/1558238090 shutdown_connections 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.008+0000 7fc2847fb700 1 --2- 172.21.15.122:0/1558238090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc27c06c3d0 0x7fc27c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.008+0000 7fc2847fb700 1 --2- 172.21.15.122:0/1558238090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc27c109500 0x7fc27c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.008+0000 7fc2847fb700 1 --2- 172.21.15.122:0/1558238090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc27c06ba10 0x7fc27c06be90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.008+0000 7fc2847fb700 1 -- 172.21.15.122:0/1558238090 >> 172.21.15.122:0/1558238090 conn(0x7fc27c079310 msgr2=0x7fc27c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.008+0000 7fc2847fb700 1 -- 172.21.15.122:0/1558238090 shutdown_connections 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.008+0000 7fc2847fb700 1 -- 172.21.15.122:0/1558238090 wait complete. 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.008+0000 7fc2847fb700 1 Processor -- start 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.008+0000 7fc2847fb700 1 -- start start 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc2847fb700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc27c06ba10 0x7fc27c102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc2847fb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc27c06c3d0 0x7fc27c1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc2847fb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc27c109500 0x7fc27c102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc2847fb700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c113610 con 0x7fc27c06ba10 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc2847fb700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c113490 con 0x7fc27c109500 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc2847fb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c113790 con 0x7fc27c06c3d0 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc282597700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc27c06ba10 0x7fc27c102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc282597700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc27c06ba10 0x7fc27c102460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49232/0 (socket says 172.21.15.122:49232) 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc282597700 1 -- 172.21.15.122:0/3735257125 learned_addr learned my addr 172.21.15.122:0/3735257125 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc281d96700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc27c06c3d0 0x7fc27c1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc282597700 1 -- 172.21.15.122:0/3735257125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc27c06c3d0 msgr2=0x7fc27c1029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc282597700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc27c06c3d0 0x7fc27c1029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc282597700 1 -- 172.21.15.122:0/3735257125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc27c109500 msgr2=0x7fc27c102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc282597700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc27c109500 0x7fc27c102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc282597700 1 -- 172.21.15.122:0/3735257125 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27c1a8010 con 0x7fc27c06ba10 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc281d96700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc27c06c3d0 0x7fc27c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:54:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc282597700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc27c06ba10 0x7fc27c102460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc27800d950 tx=0x7fc27800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc2737fe700 1 -- 172.21.15.122:0/3735257125 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278016070 con 0x7fc27c06ba10 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc2737fe700 1 -- 172.21.15.122:0/3735257125 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc278010460 con 0x7fc27c06ba10 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc2847fb700 1 -- 172.21.15.122:0/3735257125 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27c1a8300 con 0x7fc27c06ba10 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc2737fe700 1 -- 172.21.15.122:0/3735257125 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27800f620 con 0x7fc27c06ba10 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.009+0000 7fc2847fb700 1 -- 172.21.15.122:0/3735257125 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27c1a8840 con 0x7fc27c06ba10 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.010+0000 7fc2847fb700 1 -- 172.21.15.122:0/3735257125 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc27c063bb0 con 0x7fc27c06ba10 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.012+0000 7fc2737fe700 1 -- 172.21.15.122:0/3735257125 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc278019070 con 0x7fc27c06ba10 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.012+0000 7fc2737fe700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc2680b2910 0x7fc2680b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.012+0000 7fc281d96700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc2680b2910 0x7fc2680b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.012+0000 7fc2737fe700 1 -- 172.21.15.122:0/3735257125 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fc2780e23c0 con 0x7fc27c06ba10 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.013+0000 7fc2737fe700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549] conn(0x7fc2680bc350 0x7fc2680be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.013+0000 7fc2737fe700 1 -- 172.21.15.122:0/3735257125 --> [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2680beec0 con 0x7fc2680bc350 2024-05-03T17:54:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.013+0000 7fc2737fe700 1 -- 172.21.15.122:0/3735257125 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fc2780e28a0 con 0x7fc27c06ba10 2024-05-03T17:54:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.013+0000 7fc281d96700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc2680b2910 0x7fc2680b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc26c000c00 tx=0x7fc26c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.013+0000 7fc282d98700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549] conn(0x7fc2680bc350 0x7fc2680be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.013+0000 7fc282d98700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549] conn(0x7fc2680bc350 0x7fc2680be7f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.014+0000 7fc2737fe700 1 -- 172.21.15.122:0/3735257125 <== osd.7 v2:172.21.15.149:6816/139747549 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2680beec0 con 0x7fc2680bc350 2024-05-03T17:54:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.031+0000 7fc2847fb700 1 -- 172.21.15.122:0/3735257125 --> [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc27c064030 con 0x7fc2680bc350 2024-05-03T17:54:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.031+0000 7fc2737fe700 1 -- 172.21.15.122:0/3735257125 <== osd.7 v2:172.21.15.149:6816/139747549 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc27c064030 con 0x7fc2680bc350 2024-05-03T17:54:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.032+0000 7fc2847fb700 1 -- 172.21.15.122:0/3735257125 >> [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549] conn(0x7fc2680bc350 msgr2=0x7fc2680be7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.032+0000 7fc2847fb700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549] conn(0x7fc2680bc350 0x7fc2680be7f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.032+0000 7fc2847fb700 1 -- 172.21.15.122:0/3735257125 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc2680b2910 msgr2=0x7fc2680b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.032+0000 7fc2847fb700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc2680b2910 0x7fc2680b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc26c000c00 tx=0x7fc26c005fb0 comp rx=0 tx=0).stop 2024-05-03T17:54:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.032+0000 7fc2847fb700 1 -- 172.21.15.122:0/3735257125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc27c06ba10 msgr2=0x7fc27c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.032+0000 7fc2847fb700 1 --2- 172.21.15.122:0/3735257125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc27c06ba10 0x7fc27c102460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc27800d950 tx=0x7fc27800dd10 comp rx=0 tx=0).stop 2024-05-03T17:54:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.032+0000 7fc282d98700 1 -- 172.21.15.122:0/3735257125 reap_dead start 2024-05-03T17:54:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.032+0000 7fc2847fb700 1 -- 172.21.15.122:0/3735257125 shutdown_connections 2024-05-03T17:54:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.032+0000 7fc2847fb700 1 -- 172.21.15.122:0/3735257125 >> 172.21.15.122:0/3735257125 conn(0x7fc27c079310 msgr2=0x7fc27c100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.032+0000 7fc2847fb700 1 -- 172.21.15.122:0/3735257125 shutdown_connections 2024-05-03T17:54:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:13.032+0000 7fc2847fb700 1 -- 172.21.15.122:0/3735257125 wait complete. 2024-05-03T17:54:13.137 INFO:teuthology.orchestra.run.smithi122.stdout:77309411409 2024-05-03T17:54:13.138 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-03T17:54:14.023 INFO:teuthology.orchestra.run.smithi122.stdout:55834574935 2024-05-03T17:54:14.023 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-03T17:54:14.544 INFO:teuthology.orchestra.run.smithi122.stdout:274877906984 2024-05-03T17:54:14.544 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-03T17:54:14.565 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:14 smithi122 ceph-mon[29744]: pgmap v336: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:14.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:14 smithi192 ceph-mon[33753]: pgmap v336: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:14.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:14 smithi149 ceph-mon[33409]: pgmap v336: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:15.190 INFO:teuthology.orchestra.run.smithi122.stdout:227633266739 2024-05-03T17:54:15.190 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-03T17:54:15.777 INFO:teuthology.orchestra.run.smithi122.stdout:201863462968 2024-05-03T17:54:15.777 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-03T17:54:16.558 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:16 smithi122 ceph-mon[29744]: pgmap v337: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:16.584 INFO:teuthology.orchestra.run.smithi122.stdout:34359738459 2024-05-03T17:54:16.584 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-03T17:54:16.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:16 smithi192 ceph-mon[33753]: pgmap v337: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:16.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:16 smithi149 ceph-mon[33409]: pgmap v337: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:17.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 -- 172.21.15.122:0/4058609042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103eb0 msgr2=0x7fc2c4110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:17.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 --2- 172.21.15.122:0/4058609042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103eb0 0x7fc2c4110390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00b0a0 tx=0x7fc2bc00b3b0 comp rx=0 tx=0).stop 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 -- 172.21.15.122:0/4058609042 shutdown_connections 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 --2- 172.21.15.122:0/4058609042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103eb0 0x7fc2c4110390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 --2- 172.21.15.122:0/4058609042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2c4109600 0x7fc2c41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 --2- 172.21.15.122:0/4058609042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2c41034f0 0x7fc2c4103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 -- 172.21.15.122:0/4058609042 >> 172.21.15.122:0/4058609042 conn(0x7fc2c407a650 msgr2=0x7fc2c407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 -- 172.21.15.122:0/4058609042 shutdown_connections 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 -- 172.21.15.122:0/4058609042 wait complete. 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 Processor -- start 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 -- start start 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c41034f0 0x7fc2c419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2c4103eb0 0x7fc2c419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2c4109600 0x7fc2c41998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4113650 con 0x7fc2c41034f0 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41134d0 con 0x7fc2c4109600 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.590+0000 7fc2cbd82700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41137d0 con 0x7fc2c4103eb0 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2c9b1e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c41034f0 0x7fc2c419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2c9b1e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c41034f0 0x7fc2c419f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49240/0 (socket says 172.21.15.122:49240) 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2c9b1e700 1 -- 172.21.15.122:0/3767545293 learned_addr learned my addr 172.21.15.122:0/3767545293 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2c931d700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2c4103eb0 0x7fc2c419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2ca31f700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2c4109600 0x7fc2c41998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2c9b1e700 1 -- 172.21.15.122:0/3767545293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2c4103eb0 msgr2=0x7fc2c419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2c9b1e700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2c4103eb0 0x7fc2c419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2c9b1e700 1 -- 172.21.15.122:0/3767545293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2c4109600 msgr2=0x7fc2c41998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2c9b1e700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2c4109600 0x7fc2c41998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2c9b1e700 1 -- 172.21.15.122:0/3767545293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c419a050 con 0x7fc2c41034f0 2024-05-03T17:54:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2ca31f700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2c4109600 0x7fc2c41998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2c9b1e700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c41034f0 0x7fc2c419f6e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000d950 tx=0x7fc2c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2baffd700 1 -- 172.21.15.122:0/3767545293 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0016070 con 0x7fc2c41034f0 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2cbd82700 1 -- 172.21.15.122:0/3767545293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c419a340 con 0x7fc2c41034f0 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2baffd700 1 -- 172.21.15.122:0/3767545293 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c0009df0 con 0x7fc2c41034f0 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2baffd700 1 -- 172.21.15.122:0/3767545293 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000f3c0 con 0x7fc2c41034f0 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.591+0000 7fc2cbd82700 1 -- 172.21.15.122:0/3767545293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c41a65b0 con 0x7fc2c41034f0 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.592+0000 7fc2cbd82700 1 -- 172.21.15.122:0/3767545293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc2c4063c30 con 0x7fc2c41034f0 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.593+0000 7fc2baffd700 1 -- 172.21.15.122:0/3767545293 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc2c0019070 con 0x7fc2c41034f0 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.594+0000 7fc2baffd700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc2b00b2910 0x7fc2b00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.594+0000 7fc2c931d700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc2b00b2910 0x7fc2b00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.594+0000 7fc2baffd700 1 -- 172.21.15.122:0/3767545293 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fc2c00e2330 con 0x7fc2c41034f0 2024-05-03T17:54:17.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.594+0000 7fc2baffd700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209] conn(0x7fc2b00bc350 0x7fc2b00be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:17.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.594+0000 7fc2baffd700 1 -- 172.21.15.122:0/3767545293 --> [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2b00beec0 con 0x7fc2b00bc350 2024-05-03T17:54:17.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.594+0000 7fc2baffd700 1 -- 172.21.15.122:0/3767545293 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fc2c00e2810 con 0x7fc2c41034f0 2024-05-03T17:54:17.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.594+0000 7fc2c931d700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc2b00b2910 0x7fc2b00b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc2c407ab50 tx=0x7fc2b40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:17.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.595+0000 7fc2ca31f700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209] conn(0x7fc2b00bc350 0x7fc2b00be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:17.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.595+0000 7fc2ca31f700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209] conn(0x7fc2b00bc350 0x7fc2b00be7f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:17.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.596+0000 7fc2baffd700 1 -- 172.21.15.122:0/3767545293 <== osd.11 v2:172.21.15.192:6824/2098474209 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2b00beec0 con 0x7fc2b00bc350 2024-05-03T17:54:17.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.613+0000 7fc2cbd82700 1 -- 172.21.15.122:0/3767545293 --> [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc2c4064060 con 0x7fc2b00bc350 2024-05-03T17:54:17.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.613+0000 7fc2baffd700 1 -- 172.21.15.122:0/3767545293 <== osd.11 v2:172.21.15.192:6824/2098474209 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc2c4064060 con 0x7fc2b00bc350 2024-05-03T17:54:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.613+0000 7fc2cbd82700 1 -- 172.21.15.122:0/3767545293 >> [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209] conn(0x7fc2b00bc350 msgr2=0x7fc2b00be7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.613+0000 7fc2cbd82700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209] conn(0x7fc2b00bc350 0x7fc2b00be7f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.613+0000 7fc2cbd82700 1 -- 172.21.15.122:0/3767545293 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc2b00b2910 msgr2=0x7fc2b00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.613+0000 7fc2cbd82700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc2b00b2910 0x7fc2b00b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc2c407ab50 tx=0x7fc2b40073d0 comp rx=0 tx=0).stop 2024-05-03T17:54:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.614+0000 7fc2cbd82700 1 -- 172.21.15.122:0/3767545293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c41034f0 msgr2=0x7fc2c419f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.614+0000 7fc2cbd82700 1 --2- 172.21.15.122:0/3767545293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c41034f0 0x7fc2c419f6e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000d950 tx=0x7fc2c000dc60 comp rx=0 tx=0).stop 2024-05-03T17:54:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.614+0000 7fc2ca31f700 1 -- 172.21.15.122:0/3767545293 reap_dead start 2024-05-03T17:54:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.614+0000 7fc2cbd82700 1 -- 172.21.15.122:0/3767545293 shutdown_connections 2024-05-03T17:54:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.614+0000 7fc2cbd82700 1 -- 172.21.15.122:0/3767545293 >> 172.21.15.122:0/3767545293 conn(0x7fc2c407a650 msgr2=0x7fc2c4101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.614+0000 7fc2cbd82700 1 -- 172.21.15.122:0/3767545293 shutdown_connections 2024-05-03T17:54:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:17.614+0000 7fc2cbd82700 1 -- 172.21.15.122:0/3767545293 wait complete. 2024-05-03T17:54:18.397 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:18.427 INFO:teuthology.orchestra.run.smithi122.stdout:296352743461 2024-05-03T17:54:18.428 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-03T17:54:18.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:18 smithi192 ceph-mon[33753]: pgmap v338: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:18.665 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:18 smithi122 ceph-mon[29744]: pgmap v338: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:18.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:18 smithi149 ceph-mon[33409]: pgmap v338: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:18.925 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:18.925 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:18.925 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:18.928 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:18.928 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:18.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.949+0000 7f1992193700 1 -- 172.21.15.122:0/3538361752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f198c109ad0 msgr2=0x7f198c109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:18.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.949+0000 7f1992193700 1 --2- 172.21.15.122:0/3538361752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f198c109ad0 0x7f198c109f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1980009990 tx=0x7f1980009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:18.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.949+0000 7f1992193700 1 -- 172.21.15.122:0/3538361752 shutdown_connections 2024-05-03T17:54:18.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.949+0000 7f1992193700 1 --2- 172.21.15.122:0/3538361752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f198c10a490 0x7f198c1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:18.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.949+0000 7f1992193700 1 --2- 172.21.15.122:0/3538361752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f198c10fb60 0x7f198c10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:18.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.949+0000 7f1992193700 1 --2- 172.21.15.122:0/3538361752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f198c109ad0 0x7f198c109f50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:18.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.949+0000 7f1992193700 1 -- 172.21.15.122:0/3538361752 >> 172.21.15.122:0/3538361752 conn(0x7f198c076c70 msgr2=0x7f198c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:18.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.950+0000 7f1992193700 1 -- 172.21.15.122:0/3538361752 shutdown_connections 2024-05-03T17:54:18.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.951+0000 7f1992193700 1 -- 172.21.15.122:0/3538361752 wait complete. 2024-05-03T17:54:18.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.951+0000 7f1992193700 1 Processor -- start 2024-05-03T17:54:18.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.952+0000 7f1992193700 1 -- start start 2024-05-03T17:54:18.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.952+0000 7f1992193700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f198c109ad0 0x7f198c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:18.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.952+0000 7f1992193700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f198c10a490 0x7f198c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:18.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.952+0000 7f1992193700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f198c10fb60 0x7f198c06e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:18.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.952+0000 7f1992193700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f198c119d20 con 0x7f198c10a490 2024-05-03T17:54:18.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.952+0000 7f1992193700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f198c119ba0 con 0x7f198c109ad0 2024-05-03T17:54:18.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.952+0000 7f1992193700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f198c119ea0 con 0x7f198c10fb60 2024-05-03T17:54:18.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.952+0000 7f198affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f198c10a490 0x7f198c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:18.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.952+0000 7f198affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f198c10a490 0x7f198c06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33760/0 (socket says 172.21.15.122:33760) 2024-05-03T17:54:18.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.952+0000 7f198affd700 1 -- 172.21.15.122:0/2684892843 learned_addr learned my addr 172.21.15.122:0/2684892843 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:18.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.953+0000 7f198bfff700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f198c10fb60 0x7f198c06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:18.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.953+0000 7f198b7fe700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f198c109ad0 0x7f198c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:18.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.953+0000 7f198affd700 1 -- 172.21.15.122:0/2684892843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f198c10fb60 msgr2=0x7f198c06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:18.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.953+0000 7f198affd700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f198c10fb60 0x7f198c06e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:18.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.953+0000 7f198affd700 1 -- 172.21.15.122:0/2684892843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f198c109ad0 msgr2=0x7f198c074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:18.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.953+0000 7f198affd700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f198c109ad0 0x7f198c074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:18.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.953+0000 7f198affd700 1 -- 172.21.15.122:0/2684892843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c06e880 con 0x7f198c10a490 2024-05-03T17:54:18.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.953+0000 7f198affd700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f198c10a490 0x7f198c06dd30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1980000c00 tx=0x7f1980009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:18.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.953+0000 7f1988ff9700 1 -- 172.21.15.122:0/2684892843 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980027070 con 0x7f198c10a490 2024-05-03T17:54:18.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.954+0000 7f1988ff9700 1 -- 172.21.15.122:0/2684892843 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f198000fb90 con 0x7f198c10a490 2024-05-03T17:54:18.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.955+0000 7f1992193700 1 -- 172.21.15.122:0/2684892843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c06eb10 con 0x7f198c10a490 2024-05-03T17:54:18.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.955+0000 7f1988ff9700 1 -- 172.21.15.122:0/2684892843 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980017650 con 0x7f198c10a490 2024-05-03T17:54:18.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.955+0000 7f1992193700 1 -- 172.21.15.122:0/2684892843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c072a50 con 0x7f198c10a490 2024-05-03T17:54:18.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.956+0000 7f1992193700 1 -- 172.21.15.122:0/2684892843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f198c072ef0 con 0x7f198c10a490 2024-05-03T17:54:18.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.958+0000 7f1988ff9700 1 -- 172.21.15.122:0/2684892843 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f1980009d10 con 0x7f198c10a490 2024-05-03T17:54:18.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.959+0000 7f1988ff9700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f19740b2860 0x7f19740b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:18.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.960+0000 7f1988ff9700 1 -- 172.21.15.122:0/2684892843 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f19800e3660 con 0x7f198c10a490 2024-05-03T17:54:18.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.960+0000 7f198b7fe700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f19740b2860 0x7f19740b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:18.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.960+0000 7f1988ff9700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998] conn(0x7f19740bc2a0 0x7f19740be740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:18.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.960+0000 7f1988ff9700 1 -- 172.21.15.122:0/2684892843 --> [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1980010ea0 con 0x7f19740bc2a0 2024-05-03T17:54:18.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.960+0000 7f1988ff9700 1 -- 172.21.15.122:0/2684892843 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f19800e3b40 con 0x7f198c10a490 2024-05-03T17:54:18.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.960+0000 7f198bfff700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998] conn(0x7f19740bc2a0 0x7f19740be740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:18.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.961+0000 7f198b7fe700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f19740b2860 0x7f19740b4d20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f197c009bd0 tx=0x7f197c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:18.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.961+0000 7f198bfff700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998] conn(0x7f19740bc2a0 0x7f19740be740 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:18.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.962+0000 7f1988ff9700 1 -- 172.21.15.122:0/2684892843 <== osd.6 v2:172.21.15.122:6818/3419421998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1980010ea0 con 0x7f19740bc2a0 2024-05-03T17:54:18.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.979+0000 7f1992193700 1 -- 172.21.15.122:0/2684892843 --> [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f198c073370 con 0x7f19740bc2a0 2024-05-03T17:54:18.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.979+0000 7f1988ff9700 1 -- 172.21.15.122:0/2684892843 <== osd.6 v2:172.21.15.122:6818/3419421998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f198c073370 con 0x7f19740bc2a0 2024-05-03T17:54:18.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.980+0000 7f1992193700 1 -- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998] conn(0x7f19740bc2a0 msgr2=0x7f19740be740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:18.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.980+0000 7f1992193700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998] conn(0x7f19740bc2a0 0x7f19740be740 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:18.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.980+0000 7f1992193700 1 -- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f19740b2860 msgr2=0x7f19740b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:18.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.980+0000 7f1992193700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f19740b2860 0x7f19740b4d20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f197c009bd0 tx=0x7f197c009260 comp rx=0 tx=0).stop 2024-05-03T17:54:18.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.980+0000 7f1992193700 1 -- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f198c10a490 msgr2=0x7f198c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:18.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.980+0000 7f1992193700 1 --2- 172.21.15.122:0/2684892843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f198c10a490 0x7f198c06dd30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1980000c00 tx=0x7f1980009510 comp rx=0 tx=0).stop 2024-05-03T17:54:18.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.980+0000 7f198bfff700 1 -- 172.21.15.122:0/2684892843 reap_dead start 2024-05-03T17:54:18.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.980+0000 7f1992193700 1 -- 172.21.15.122:0/2684892843 shutdown_connections 2024-05-03T17:54:18.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.980+0000 7f1992193700 1 -- 172.21.15.122:0/2684892843 >> 172.21.15.122:0/2684892843 conn(0x7f198c076c70 msgr2=0x7f198c077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:18.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.980+0000 7f1992193700 1 -- 172.21.15.122:0/2684892843 shutdown_connections 2024-05-03T17:54:18.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:18.980+0000 7f1992193700 1 -- 172.21.15.122:0/2684892843 wait complete. 2024-05-03T17:54:20.555 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:20 smithi122 ceph-mon[29744]: pgmap v339: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s 2024-05-03T17:54:20.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:20 smithi192 ceph-mon[33753]: pgmap v339: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s 2024-05-03T17:54:20.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:20 smithi149 ceph-mon[33409]: pgmap v339: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s 2024-05-03T17:54:20.888 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:20.889 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:20.889 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:21.319 INFO:teuthology.orchestra.run.smithi122.stdout:180388626495 2024-05-03T17:54:21.320 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-03T17:54:22.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:22 smithi122 ceph-mon[29744]: pgmap v340: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-03T17:54:22.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:22 smithi192 ceph-mon[33753]: pgmap v340: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-03T17:54:22.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:22 smithi149 ceph-mon[33409]: pgmap v340: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-03T17:54:24.453 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:24 smithi122 ceph-mon[29744]: pgmap v341: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:24.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:24 smithi192 ceph-mon[33753]: pgmap v341: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:24.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:24 smithi149 ceph-mon[33409]: pgmap v341: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:26.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.096+0000 7f2a015cc700 1 -- 172.21.15.122:0/3484301299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29fc108400 msgr2=0x7f29fc1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:26.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.096+0000 7f2a015cc700 1 --2- 172.21.15.122:0/3484301299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29fc108400 0x7f29fc1087e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f29ec009990 tx=0x7f29ec009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:26.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.096+0000 7f2a015cc700 1 -- 172.21.15.122:0/3484301299 shutdown_connections 2024-05-03T17:54:26.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.096+0000 7f2a015cc700 1 --2- 172.21.15.122:0/3484301299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29fc102d50 0x7f29fc10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:26.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.096+0000 7f2a015cc700 1 --2- 172.21.15.122:0/3484301299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29fc108400 0x7f29fc1087e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:26.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.096+0000 7f2a015cc700 1 --2- 172.21.15.122:0/3484301299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29fc102390 0x7f29fc102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:26.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.096+0000 7f2a015cc700 1 -- 172.21.15.122:0/3484301299 >> 172.21.15.122:0/3484301299 conn(0x7f29fc079310 msgr2=0x7f29fc079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:26.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.096+0000 7f2a015cc700 1 -- 172.21.15.122:0/3484301299 shutdown_connections 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.096+0000 7f2a015cc700 1 -- 172.21.15.122:0/3484301299 wait complete. 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f2a015cc700 1 Processor -- start 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f2a015cc700 1 -- start start 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f2a015cc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29fc102390 0x7f29fc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f2a015cc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29fc102d50 0x7f29fc19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f2a015cc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29fc108400 0x7f29fc1a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f2a015cc700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc112550 con 0x7f29fc102390 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f2a015cc700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1123d0 con 0x7f29fc102d50 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f2a015cc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1126d0 con 0x7f29fc108400 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f29faffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29fc102390 0x7f29fc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f29faffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29fc102390 0x7f29fc19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33778/0 (socket says 172.21.15.122:33778) 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f29faffd700 1 -- 172.21.15.122:0/1115452413 learned_addr learned my addr 172.21.15.122:0/1115452413 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:26.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f29fb7fe700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29fc108400 0x7f29fc1a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f29f27fc700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29fc102d50 0x7f29fc19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f29faffd700 1 -- 172.21.15.122:0/1115452413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29fc108400 msgr2=0x7f29fc1a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f29faffd700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29fc108400 0x7f29fc1a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f29faffd700 1 -- 172.21.15.122:0/1115452413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29fc102d50 msgr2=0x7f29fc19af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f29faffd700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29fc102d50 0x7f29fc19af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f29faffd700 1 -- 172.21.15.122:0/1115452413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc1a5fb0 con 0x7f29fc102390 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.097+0000 7f29f27fc700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29fc102d50 0x7f29fc19af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.098+0000 7f29faffd700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29fc102390 0x7f29fc19a9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f29ec000c00 tx=0x7f29ec009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.098+0000 7f29f8ff9700 1 -- 172.21.15.122:0/1115452413 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec027070 con 0x7f29fc102390 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.098+0000 7f2a015cc700 1 -- 172.21.15.122:0/1115452413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29fc1a6180 con 0x7f29fc102390 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.098+0000 7f29f8ff9700 1 -- 172.21.15.122:0/1115452413 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ec022470 con 0x7f29fc102390 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.098+0000 7f29f8ff9700 1 -- 172.21.15.122:0/1115452413 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec017650 con 0x7f29fc102390 2024-05-03T17:54:26.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.098+0000 7f2a015cc700 1 -- 172.21.15.122:0/1115452413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29fc1a6630 con 0x7f29fc102390 2024-05-03T17:54:26.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.099+0000 7f2a015cc700 1 -- 172.21.15.122:0/1115452413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29fc06a190 con 0x7f29fc102390 2024-05-03T17:54:26.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.100+0000 7f29f8ff9700 1 -- 172.21.15.122:0/1115452413 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f29ec02a070 con 0x7f29fc102390 2024-05-03T17:54:26.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.101+0000 7f29f8ff9700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29e80b2910 0x7f29e80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:26.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.101+0000 7f29f27fc700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29e80b2910 0x7f29e80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:26.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.101+0000 7f29f8ff9700 1 -- 172.21.15.122:0/1115452413 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f29ec0e35c0 con 0x7f29fc102390 2024-05-03T17:54:26.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.101+0000 7f29f27fc700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29e80b2910 0x7f29e80b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f29fc079960 tx=0x7f29e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:26.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.105+0000 7f29f8ff9700 1 -- 172.21.15.122:0/1115452413 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f29ec013070 con 0x7f29fc102390 2024-05-03T17:54:26.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.320+0000 7f2a015cc700 1 -- 172.21.15.122:0/1115452413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f29fc02e510 con 0x7f29fc102390 2024-05-03T17:54:26.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.320+0000 7f29f8ff9700 1 -- 172.21.15.122:0/1115452413 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f29ec0ab310 con 0x7f29fc102390 2024-05-03T17:54:26.315 INFO:teuthology.orchestra.run.smithi122.stdout:77309411412 2024-05-03T17:54:26.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 -- 172.21.15.122:0/1115452413 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29e80b2910 msgr2=0x7f29e80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:26.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29e80b2910 0x7f29e80b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f29fc079960 tx=0x7f29e4009260 comp rx=0 tx=0).stop 2024-05-03T17:54:26.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 -- 172.21.15.122:0/1115452413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29fc102390 msgr2=0x7f29fc19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:26.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29fc102390 0x7f29fc19a9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f29ec000c00 tx=0x7f29ec009580 comp rx=0 tx=0).stop 2024-05-03T17:54:26.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 -- 172.21.15.122:0/1115452413 shutdown_connections 2024-05-03T17:54:26.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29e80b2910 0x7f29e80b4dd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:26.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29fc108400 0x7f29fc1a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:26.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29fc102390 0x7f29fc19a9f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:26.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 --2- 172.21.15.122:0/1115452413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29fc102d50 0x7f29fc19af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:26.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 -- 172.21.15.122:0/1115452413 >> 172.21.15.122:0/1115452413 conn(0x7f29fc079310 msgr2=0x7f29fc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:26.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 -- 172.21.15.122:0/1115452413 shutdown_connections 2024-05-03T17:54:26.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:26.322+0000 7f2a015cc700 1 -- 172.21.15.122:0/1115452413 wait complete. 2024-05-03T17:54:26.573 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:26 smithi122 ceph-mon[29744]: pgmap v342: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:26.573 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:26 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1115452413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T17:54:26.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:26 smithi192 ceph-mon[33753]: pgmap v342: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:26.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:26 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/1115452413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T17:54:26.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:26 smithi149 ceph-mon[33409]: pgmap v342: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:26.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:26 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/1115452413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T17:54:28.299 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411409 got 77309411412 for osd.2 2024-05-03T17:54:28.299 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:28.553 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:28 smithi122 ceph-mon[29744]: pgmap v343: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:54:28.553 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:28 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3198376002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T17:54:28.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:28 smithi192 ceph-mon[33753]: pgmap v343: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:54:28.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:28 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3198376002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T17:54:28.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:28 smithi149 ceph-mon[33409]: pgmap v343: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:54:28.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:28 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3198376002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T17:54:29.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.120+0000 7facabfb7700 1 -- 172.21.15.122:0/3721887714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faca4109510 msgr2=0x7faca41098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:29.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.120+0000 7facabfb7700 1 --2- 172.21.15.122:0/3721887714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faca4109510 0x7faca41098f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faca0009990 tx=0x7faca0009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.120+0000 7facabfb7700 1 -- 172.21.15.122:0/3721887714 shutdown_connections 2024-05-03T17:54:29.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.120+0000 7facabfb7700 1 --2- 172.21.15.122:0/3721887714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faca406bb40 0x7faca4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.120+0000 7facabfb7700 1 --2- 172.21.15.122:0/3721887714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faca4109510 0x7faca41098f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.120+0000 7facabfb7700 1 --2- 172.21.15.122:0/3721887714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca406b180 0x7faca406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.120+0000 7facabfb7700 1 -- 172.21.15.122:0/3721887714 >> 172.21.15.122:0/3721887714 conn(0x7faca4079350 msgr2=0x7faca4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:29.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.120+0000 7facabfb7700 1 -- 172.21.15.122:0/3721887714 shutdown_connections 2024-05-03T17:54:29.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.120+0000 7facabfb7700 1 -- 172.21.15.122:0/3721887714 wait complete. 2024-05-03T17:54:29.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.121+0000 7facabfb7700 1 Processor -- start 2024-05-03T17:54:29.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.121+0000 7facabfb7700 1 -- start start 2024-05-03T17:54:29.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.121+0000 7facabfb7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca406b180 0x7faca419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:29.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.121+0000 7facabfb7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faca406bb40 0x7faca419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:29.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.121+0000 7facabfb7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faca4109510 0x7faca41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:29.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.121+0000 7facabfb7700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faca4113620 con 0x7faca4109510 2024-05-03T17:54:29.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.121+0000 7facabfb7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faca41134a0 con 0x7faca406bb40 2024-05-03T17:54:29.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.121+0000 7facabfb7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faca41137a0 con 0x7faca406b180 2024-05-03T17:54:29.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.121+0000 7facaa554700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faca4109510 0x7faca41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:29.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.121+0000 7facaa554700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faca4109510 0x7faca41999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49208/0 (socket says 172.21.15.122:49208) 2024-05-03T17:54:29.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.121+0000 7facaa554700 1 -- 172.21.15.122:0/2193586744 learned_addr learned my addr 172.21.15.122:0/2193586744 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:29.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7faca9552700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faca406bb40 0x7faca419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7faca9d53700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca406b180 0x7faca419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7facaa554700 1 -- 172.21.15.122:0/2193586744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca406b180 msgr2=0x7faca419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7facaa554700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca406b180 0x7faca419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7facaa554700 1 -- 172.21.15.122:0/2193586744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faca406bb40 msgr2=0x7faca419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7facaa554700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faca406bb40 0x7faca419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7facaa554700 1 -- 172.21.15.122:0/2193586744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca419a100 con 0x7faca4109510 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7faca9552700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faca406bb40 0x7faca419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7facaa554700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faca4109510 0x7faca41999a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fac9c007ee0 tx=0x7fac9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7fac9affd700 1 -- 172.21.15.122:0/2193586744 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00ddd0 con 0x7faca4109510 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7fac9affd700 1 -- 172.21.15.122:0/2193586744 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac9c004020 con 0x7faca4109510 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7fac9affd700 1 -- 172.21.15.122:0/2193586744 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c015660 con 0x7faca4109510 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7facabfb7700 1 -- 172.21.15.122:0/2193586744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca419a3f0 con 0x7faca4109510 2024-05-03T17:54:29.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.122+0000 7facabfb7700 1 -- 172.21.15.122:0/2193586744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca41a66b0 con 0x7faca4109510 2024-05-03T17:54:29.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.123+0000 7facabfb7700 1 -- 172.21.15.122:0/2193586744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca410b9f0 con 0x7faca4109510 2024-05-03T17:54:29.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.124+0000 7fac9affd700 1 -- 172.21.15.122:0/2193586744 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fac9c0041d0 con 0x7faca4109510 2024-05-03T17:54:29.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.125+0000 7fac9affd700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac900b2990 0x7fac900b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:29.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.125+0000 7faca9d53700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac900b2990 0x7fac900b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:29.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.125+0000 7fac9affd700 1 -- 172.21.15.122:0/2193586744 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fac9c0e2470 con 0x7faca4109510 2024-05-03T17:54:29.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.126+0000 7faca9d53700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac900b2990 0x7fac900b4e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faca000b5c0 tx=0x7faca0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:29.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.129+0000 7fac9affd700 1 -- 172.21.15.122:0/2193586744 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fac9c010070 con 0x7faca4109510 2024-05-03T17:54:29.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 -- 172.21.15.122:0/160684753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6168108850 msgr2=0x7f61681103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:29.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 --2- 172.21.15.122:0/160684753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6168108850 0x7f61681103d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f616000b0d0 tx=0x7f616000b3e0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 -- 172.21.15.122:0/160684753 shutdown_connections 2024-05-03T17:54:29.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 --2- 172.21.15.122:0/160684753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6168108850 0x7f61681103d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 --2- 172.21.15.122:0/160684753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6168103750 0x7f6168103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 --2- 172.21.15.122:0/160684753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6168104100 0x7f6168108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 -- 172.21.15.122:0/160684753 >> 172.21.15.122:0/160684753 conn(0x7f616807a600 msgr2=0x7f616807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:29.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 -- 172.21.15.122:0/160684753 shutdown_connections 2024-05-03T17:54:29.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 -- 172.21.15.122:0/160684753 wait complete. 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 Processor -- start 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 -- start start 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6168103750 0x7f61680757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6168104100 0x7f6168075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6168108850 0x7f6168076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.770+0000 7f617045b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6168113620 con 0x7f6168104100 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f617045b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f61681134a0 con 0x7f6168103750 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f617045b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f61681137a0 con 0x7f6168108850 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616d9f6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6168104100 0x7f6168075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616d9f6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6168104100 0x7f6168075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33796/0 (socket says 172.21.15.122:33796) 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616d9f6700 1 -- 172.21.15.122:0/3198376002 learned_addr learned my addr 172.21.15.122:0/3198376002 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616e9f8700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6168108850 0x7f6168076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:29.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616e1f7700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6168103750 0x7f61680757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616d9f6700 1 -- 172.21.15.122:0/3198376002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6168108850 msgr2=0x7f6168076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616d9f6700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6168108850 0x7f6168076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616d9f6700 1 -- 172.21.15.122:0/3198376002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6168103750 msgr2=0x7f61680757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616d9f6700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6168103750 0x7f61680757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616d9f6700 1 -- 172.21.15.122:0/3198376002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61681a7f60 con 0x7f6168104100 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616e9f8700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6168108850 0x7f6168076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616e1f7700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6168103750 0x7f61680757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f616d9f6700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6168104100 0x7f6168075ce0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f615800c9b0 tx=0x7f615800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f615f7fe700 1 -- 172.21.15.122:0/3198376002 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158007b00 con 0x7f6168104100 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f615f7fe700 1 -- 172.21.15.122:0/3198376002 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6158007ca0 con 0x7f6168104100 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f617045b700 1 -- 172.21.15.122:0/3198376002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61681a8250 con 0x7f6168104100 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f615f7fe700 1 -- 172.21.15.122:0/3198376002 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158018730 con 0x7f6168104100 2024-05-03T17:54:29.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.771+0000 7f617045b700 1 -- 172.21.15.122:0/3198376002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61681a87e0 con 0x7f6168104100 2024-05-03T17:54:29.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.772+0000 7f617045b700 1 -- 172.21.15.122:0/3198376002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f616806a190 con 0x7f6168104100 2024-05-03T17:54:29.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.773+0000 7f615f7fe700 1 -- 172.21.15.122:0/3198376002 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f615800f450 con 0x7f6168104100 2024-05-03T17:54:29.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.774+0000 7f615f7fe700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f61540b2990 0x7f61540b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:29.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.774+0000 7f616e1f7700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f61540b2990 0x7f61540b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:29.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.774+0000 7f615f7fe700 1 -- 172.21.15.122:0/3198376002 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f61580e2230 con 0x7f6168104100 2024-05-03T17:54:29.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.774+0000 7f616e1f7700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f61540b2990 0x7f61540b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6164005950 tx=0x7f61640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:29.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.778+0000 7f615f7fe700 1 -- 172.21.15.122:0/3198376002 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6158014030 con 0x7f6168104100 2024-05-03T17:54:29.200 INFO:teuthology.orchestra.run.smithi122.stdout:154618822726 2024-05-03T17:54:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.990+0000 7f617045b700 1 -- 172.21.15.122:0/3198376002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f616802e6d0 con 0x7f6168104100 2024-05-03T17:54:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.990+0000 7f615f7fe700 1 -- 172.21.15.122:0/3198376002 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f61580a9ef0 con 0x7f6168104100 2024-05-03T17:54:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 -- 172.21.15.122:0/3198376002 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f61540b2990 msgr2=0x7f61540b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f61540b2990 0x7f61540b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6164005950 tx=0x7f61640058e0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 -- 172.21.15.122:0/3198376002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6168104100 msgr2=0x7f6168075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6168104100 0x7f6168075ce0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f615800c9b0 tx=0x7f615800cd70 comp rx=0 tx=0).stop 2024-05-03T17:54:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 -- 172.21.15.122:0/3198376002 shutdown_connections 2024-05-03T17:54:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f61540b2990 0x7f61540b4e50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6168108850 0x7f6168076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6168103750 0x7f61680757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 --2- 172.21.15.122:0/3198376002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6168104100 0x7f6168075ce0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 -- 172.21.15.122:0/3198376002 >> 172.21.15.122:0/3198376002 conn(0x7f616807a600 msgr2=0x7f61681014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:29.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 -- 172.21.15.122:0/3198376002 shutdown_connections 2024-05-03T17:54:29.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:27.993+0000 7f617045b700 1 -- 172.21.15.122:0/3198376002 wait complete. 2024-05-03T17:54:29.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.352+0000 7facabfb7700 1 -- 172.21.15.122:0/2193586744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7faca4104bd0 con 0x7faca4109510 2024-05-03T17:54:29.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.353+0000 7fac9affd700 1 -- 172.21.15.122:0/2193586744 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7faca4104bd0 con 0x7faca4109510 2024-05-03T17:54:29.348 INFO:teuthology.orchestra.run.smithi122.stdout:274877906989 2024-05-03T17:54:29.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 -- 172.21.15.122:0/2193586744 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac900b2990 msgr2=0x7fac900b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:29.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac900b2990 0x7fac900b4e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faca000b5c0 tx=0x7faca0005960 comp rx=0 tx=0).stop 2024-05-03T17:54:29.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 -- 172.21.15.122:0/2193586744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faca4109510 msgr2=0x7faca41999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:29.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faca4109510 0x7faca41999a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fac9c007ee0 tx=0x7fac9c00d3b0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 -- 172.21.15.122:0/2193586744 shutdown_connections 2024-05-03T17:54:29.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fac900b2990 0x7fac900b4e50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faca4109510 0x7faca41999a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca406b180 0x7faca419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 --2- 172.21.15.122:0/2193586744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faca406bb40 0x7faca419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:29.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 -- 172.21.15.122:0/2193586744 >> 172.21.15.122:0/2193586744 conn(0x7faca4079350 msgr2=0x7faca41010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:29.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 -- 172.21.15.122:0/2193586744 shutdown_connections 2024-05-03T17:54:29.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.355+0000 7facabfb7700 1 -- 172.21.15.122:0/2193586744 wait complete. 2024-05-03T17:54:29.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:29 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2762030029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T17:54:29.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:29 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2762030029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T17:54:29.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:29 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2762030029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T17:54:30.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.029+0000 7fbf76794700 1 -- 172.21.15.122:0/2158371758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf70108580 msgr2=0x7fbf70108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:30.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.029+0000 7fbf76794700 1 --2- 172.21.15.122:0/2158371758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf70108580 0x7fbf70108960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009990 tx=0x7fbf60009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.029+0000 7fbf76794700 1 -- 172.21.15.122:0/2158371758 shutdown_connections 2024-05-03T17:54:30.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.029+0000 7fbf76794700 1 --2- 172.21.15.122:0/2158371758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf70102820 0x7fbf70110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.029+0000 7fbf76794700 1 --2- 172.21.15.122:0/2158371758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf70108580 0x7fbf70108960 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.029+0000 7fbf76794700 1 --2- 172.21.15.122:0/2158371758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf70101e60 0x7fbf701022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.029+0000 7fbf76794700 1 -- 172.21.15.122:0/2158371758 >> 172.21.15.122:0/2158371758 conn(0x7fbf700776c0 msgr2=0x7fbf70077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:30.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.029+0000 7fbf76794700 1 -- 172.21.15.122:0/2158371758 shutdown_connections 2024-05-03T17:54:30.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.029+0000 7fbf76794700 1 -- 172.21.15.122:0/2158371758 wait complete. 2024-05-03T17:54:30.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.030+0000 7fbf76794700 1 Processor -- start 2024-05-03T17:54:30.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.030+0000 7fbf76794700 1 -- start start 2024-05-03T17:54:30.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.030+0000 7fbf76794700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf70101e60 0x7fbf7019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.030+0000 7fbf76794700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf70102820 0x7fbf7019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.030+0000 7fbf76794700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf70108580 0x7fbf70199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.030+0000 7fbf76794700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70113690 con 0x7fbf70101e60 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.030+0000 7fbf76794700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70113510 con 0x7fbf70108580 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.030+0000 7fbf76794700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70113810 con 0x7fbf70102820 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf70101e60 0x7fbf7019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf70101e60 0x7fbf7019f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49184/0 (socket says 172.21.15.122:49184) 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6ffff700 1 -- 172.21.15.122:0/2762030029 learned_addr learned my addr 172.21.15.122:0/2762030029 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6f7fe700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf70102820 0x7fbf7019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf74d31700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf70108580 0x7fbf70199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6ffff700 1 -- 172.21.15.122:0/2762030029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf70102820 msgr2=0x7fbf7019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6ffff700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf70102820 0x7fbf7019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6ffff700 1 -- 172.21.15.122:0/2762030029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf70108580 msgr2=0x7fbf70199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6ffff700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf70108580 0x7fbf70199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6ffff700 1 -- 172.21.15.122:0/2762030029 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7019a170 con 0x7fbf70101e60 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf74d31700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf70108580 0x7fbf70199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6ffff700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf70101e60 0x7fbf7019f850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbf60000c00 tx=0x7fbf600047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6d7fa700 1 -- 172.21.15.122:0/2762030029 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60027070 con 0x7fbf70101e60 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf76794700 1 -- 172.21.15.122:0/2762030029 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7019a400 con 0x7fbf70101e60 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6d7fa700 1 -- 172.21.15.122:0/2762030029 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf60003f20 con 0x7fbf70101e60 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf6d7fa700 1 -- 172.21.15.122:0/2762030029 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60022cd0 con 0x7fbf70101e60 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.031+0000 7fbf76794700 1 -- 172.21.15.122:0/2762030029 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf70102f70 con 0x7fbf70101e60 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.032+0000 7fbf76794700 1 -- 172.21.15.122:0/2762030029 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7006b690 con 0x7fbf70101e60 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.034+0000 7fbf6d7fa700 1 -- 172.21.15.122:0/2762030029 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fbf6002a070 con 0x7fbf70101e60 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.035+0000 7fbf6d7fa700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf580b2910 0x7fbf580b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.035+0000 7fbf6f7fe700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf580b2910 0x7fbf580b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:30.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.035+0000 7fbf6d7fa700 1 -- 172.21.15.122:0/2762030029 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fbf600e34e0 con 0x7fbf70101e60 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.035+0000 7fbf6f7fe700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf580b2910 0x7fbf580b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbf70077d70 tx=0x7fbf640073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.038+0000 7fbf6d7fa700 1 -- 172.21.15.122:0/2762030029 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbf60013070 con 0x7fbf70101e60 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.253+0000 7fbf76794700 1 -- 172.21.15.122:0/2762030029 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fbf7019af40 con 0x7fbf70101e60 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.253+0000 7fbf6d7fa700 1 -- 172.21.15.122:0/2762030029 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbf600ab130 con 0x7fbf70101e60 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.255+0000 7fbf76794700 1 -- 172.21.15.122:0/2762030029 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf580b2910 msgr2=0x7fbf580b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.255+0000 7fbf76794700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf580b2910 0x7fbf580b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbf70077d70 tx=0x7fbf640073d0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.255+0000 7fbf76794700 1 -- 172.21.15.122:0/2762030029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf70101e60 msgr2=0x7fbf7019f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.255+0000 7fbf76794700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf70101e60 0x7fbf7019f850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbf60000c00 tx=0x7fbf600047b0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.256+0000 7fbf76794700 1 -- 172.21.15.122:0/2762030029 shutdown_connections 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.256+0000 7fbf76794700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf580b2910 0x7fbf580b4dd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.256+0000 7fbf76794700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf70108580 0x7fbf70199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.256+0000 7fbf76794700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf70101e60 0x7fbf7019f850 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.256+0000 7fbf76794700 1 --2- 172.21.15.122:0/2762030029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf70102820 0x7fbf7019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.256+0000 7fbf76794700 1 -- 172.21.15.122:0/2762030029 >> 172.21.15.122:0/2762030029 conn(0x7fbf700776c0 msgr2=0x7fbf70100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:30.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.256+0000 7fbf76794700 1 -- 172.21.15.122:0/2762030029 shutdown_connections 2024-05-03T17:54:30.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:29.256+0000 7fbf76794700 1 -- 172.21.15.122:0/2762030029 wait complete. 2024-05-03T17:54:30.361 INFO:teuthology.orchestra.run.smithi122.stdout:227633266742 2024-05-03T17:54:30.567 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:30.568 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:30.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:30 smithi192 ceph-mon[33753]: pgmap v344: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:30.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:30 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2193586744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T17:54:30.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:30 smithi122 ceph-mon[29744]: pgmap v344: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:30.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:30 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2193586744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T17:54:30.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:30 smithi149 ceph-mon[33409]: pgmap v344: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:30.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:30 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2193586744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T17:54:31.376 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822720 got 154618822726 for osd.5 2024-05-03T17:54:31.377 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:32.551 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:32 smithi122 ceph-mon[29744]: pgmap v345: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:32.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:32 smithi192 ceph-mon[33753]: pgmap v345: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:32.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.661+0000 7f087da65700 1 -- 172.21.15.122:0/1769155275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0878108410 msgr2=0x7f08781087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:32.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.661+0000 7f087da65700 1 --2- 172.21.15.122:0/1769155275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0878108410 0x7f08781087f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0868009990 tx=0x7f0868009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:32.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.661+0000 7f087da65700 1 -- 172.21.15.122:0/1769155275 shutdown_connections 2024-05-03T17:54:32.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.661+0000 7f087da65700 1 --2- 172.21.15.122:0/1769155275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878102d60 0x7f087810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:32.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.661+0000 7f087da65700 1 --2- 172.21.15.122:0/1769155275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0878108410 0x7f08781087f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:32.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.661+0000 7f087da65700 1 --2- 172.21.15.122:0/1769155275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08781023a0 0x7f0878102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:32.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.661+0000 7f087da65700 1 -- 172.21.15.122:0/1769155275 >> 172.21.15.122:0/1769155275 conn(0x7f0878079350 msgr2=0x7f0878079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:32.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.661+0000 7f087da65700 1 -- 172.21.15.122:0/1769155275 shutdown_connections 2024-05-03T17:54:32.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.661+0000 7f087da65700 1 -- 172.21.15.122:0/1769155275 wait complete. 2024-05-03T17:54:32.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f087da65700 1 Processor -- start 2024-05-03T17:54:32.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f087da65700 1 -- start start 2024-05-03T17:54:32.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f087da65700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08781023a0 0x7f087819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:32.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f087da65700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878102d60 0x7f087819af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:32.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f087da65700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0878108410 0x7f08781a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:32.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f087da65700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0878112530 con 0x7f0878108410 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f087da65700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f08781123b0 con 0x7f08781023a0 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f087da65700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f08781126b0 con 0x7f0878102d60 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f0877fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0878108410 0x7f08781a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f0877fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0878108410 0x7f08781a5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49226/0 (socket says 172.21.15.122:49226) 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f0877fff700 1 -- 172.21.15.122:0/3916607084 learned_addr learned my addr 172.21.15.122:0/3916607084 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f08777fe700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08781023a0 0x7f087819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f0876ffd700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878102d60 0x7f087819af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f0877fff700 1 -- 172.21.15.122:0/3916607084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878102d60 msgr2=0x7f087819af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f0877fff700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878102d60 0x7f087819af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.662+0000 7f0877fff700 1 -- 172.21.15.122:0/3916607084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08781023a0 msgr2=0x7f087819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.663+0000 7f0877fff700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08781023a0 0x7f087819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:32.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.663+0000 7f0877fff700 1 -- 172.21.15.122:0/3916607084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08781a5fb0 con 0x7f0878108410 2024-05-03T17:54:32.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.663+0000 7f0876ffd700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878102d60 0x7f087819af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:54:32.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.663+0000 7f08777fe700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08781023a0 0x7f087819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T17:54:32.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.663+0000 7f0877fff700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0878108410 0x7f08781a5a70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0870007ee0 tx=0x7f087000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:32.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.663+0000 7f0874ff9700 1 -- 172.21.15.122:0/3916607084 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087000dbd0 con 0x7f0878108410 2024-05-03T17:54:32.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.663+0000 7f087da65700 1 -- 172.21.15.122:0/3916607084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08781a6240 con 0x7f0878108410 2024-05-03T17:54:32.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.663+0000 7f0874ff9700 1 -- 172.21.15.122:0/3916607084 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087000dd70 con 0x7f0878108410 2024-05-03T17:54:32.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.663+0000 7f087da65700 1 -- 172.21.15.122:0/3916607084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08781a6750 con 0x7f0878108410 2024-05-03T17:54:32.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.663+0000 7f0874ff9700 1 -- 172.21.15.122:0/3916607084 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870005990 con 0x7f0878108410 2024-05-03T17:54:32.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.664+0000 7f087da65700 1 -- 172.21.15.122:0/3916607084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0878103ac0 con 0x7f0878108410 2024-05-03T17:54:32.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.665+0000 7f0874ff9700 1 -- 172.21.15.122:0/3916607084 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f087001a020 con 0x7f0878108410 2024-05-03T17:54:32.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.666+0000 7f0874ff9700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f08600b2910 0x7f08600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:32.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.666+0000 7f08777fe700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f08600b2910 0x7f08600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:32.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.666+0000 7f0874ff9700 1 -- 172.21.15.122:0/3916607084 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f08700e20f0 con 0x7f0878108410 2024-05-03T17:54:32.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.666+0000 7f08777fe700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f08600b2910 0x7f08600b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f086800b2e0 tx=0x7f086801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:32.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.670+0000 7f0874ff9700 1 -- 172.21.15.122:0/3916607084 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0870010070 con 0x7f0878108410 2024-05-03T17:54:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:32 smithi149 ceph-mon[33409]: pgmap v345: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:32.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.880+0000 7f087da65700 1 -- 172.21.15.122:0/3916607084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f087806b7d0 con 0x7f0878108410 2024-05-03T17:54:32.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.881+0000 7f0874ff9700 1 -- 172.21.15.122:0/3916607084 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f08700a9db0 con 0x7f0878108410 2024-05-03T17:54:32.875 INFO:teuthology.orchestra.run.smithi122.stdout:55834574938 2024-05-03T17:54:32.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.883+0000 7f087da65700 1 -- 172.21.15.122:0/3916607084 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f08600b2910 msgr2=0x7f08600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:32.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.883+0000 7f087da65700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f08600b2910 0x7f08600b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f086800b2e0 tx=0x7f086801a040 comp rx=0 tx=0).stop 2024-05-03T17:54:32.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.883+0000 7f087da65700 1 -- 172.21.15.122:0/3916607084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0878108410 msgr2=0x7f08781a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:32.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.883+0000 7f087da65700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0878108410 0x7f08781a5a70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0870007ee0 tx=0x7f087000d3b0 comp rx=0 tx=0).stop 2024-05-03T17:54:32.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.883+0000 7f087da65700 1 -- 172.21.15.122:0/3916607084 shutdown_connections 2024-05-03T17:54:32.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.883+0000 7f087da65700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f08600b2910 0x7f08600b4dd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:32.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.883+0000 7f087da65700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0878108410 0x7f08781a5a70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:32.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.883+0000 7f087da65700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08781023a0 0x7f087819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:32.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.883+0000 7f087da65700 1 --2- 172.21.15.122:0/3916607084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878102d60 0x7f087819af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:32.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.883+0000 7f087da65700 1 -- 172.21.15.122:0/3916607084 >> 172.21.15.122:0/3916607084 conn(0x7f0878079350 msgr2=0x7f0878101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:32.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.884+0000 7f087da65700 1 -- 172.21.15.122:0/3916607084 shutdown_connections 2024-05-03T17:54:32.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:32.884+0000 7f087da65700 1 -- 172.21.15.122:0/3916607084 wait complete. 2024-05-03T17:54:33.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:33 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3916607084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T17:54:33.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:33 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:54:33.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:33 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3916607084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T17:54:33.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:33 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:54:33.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:33 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3916607084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T17:54:33.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:33 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:54:34.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:34 smithi122 ceph-mon[29744]: pgmap v346: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:34.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:34 smithi192 ceph-mon[33753]: pgmap v346: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:34.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:34 smithi149 ceph-mon[33409]: pgmap v346: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:54:35.151 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:35.177 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266739 got 227633266742 for osd.8 2024-05-03T17:54:35.178 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:35.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.342+0000 7f9a53eeb700 1 -- 172.21.15.122:0/2965863958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a4c101b80 msgr2=0x7f9a4c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:35.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.342+0000 7f9a53eeb700 1 --2- 172.21.15.122:0/2965863958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a4c101b80 0x7f9a4c102000 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00aad0 tx=0x7f9a3c00ade0 comp rx=0 tx=0).stop 2024-05-03T17:54:35.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.343+0000 7f9a53eeb700 1 -- 172.21.15.122:0/2965863958 shutdown_connections 2024-05-03T17:54:35.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.343+0000 7f9a53eeb700 1 --2- 172.21.15.122:0/2965863958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c102540 0x7f9a4c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:35.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.343+0000 7f9a53eeb700 1 --2- 172.21.15.122:0/2965863958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a4c107be0 0x7f9a4c107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:35.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.343+0000 7f9a53eeb700 1 --2- 172.21.15.122:0/2965863958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a4c101b80 0x7f9a4c102000 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:35.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.343+0000 7f9a53eeb700 1 -- 172.21.15.122:0/2965863958 >> 172.21.15.122:0/2965863958 conn(0x7f9a4c078a10 msgr2=0x7f9a4c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:35.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.343+0000 7f9a53eeb700 1 -- 172.21.15.122:0/2965863958 shutdown_connections 2024-05-03T17:54:35.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.343+0000 7f9a53eeb700 1 -- 172.21.15.122:0/2965863958 wait complete. 2024-05-03T17:54:35.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.343+0000 7f9a53eeb700 1 Processor -- start 2024-05-03T17:54:35.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a53eeb700 1 -- start start 2024-05-03T17:54:35.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a53eeb700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a4c101b80 0x7f9a4c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:35.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a53eeb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a4c102540 0x7f9a4c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:35.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a53eeb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c107be0 0x7f9a4c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:35.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a53eeb700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1114b0 con 0x7f9a4c101b80 2024-05-03T17:54:35.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a53eeb700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c111330 con 0x7f9a4c102540 2024-05-03T17:54:35.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a53eeb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c111630 con 0x7f9a4c107be0 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a51c87700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a4c101b80 0x7f9a4c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a51c87700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a4c101b80 0x7f9a4c19f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49248/0 (socket says 172.21.15.122:49248) 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a51c87700 1 -- 172.21.15.122:0/557417935 learned_addr learned my addr 172.21.15.122:0/557417935 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a52488700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c107be0 0x7f9a4c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a51486700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a4c102540 0x7f9a4c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a51c87700 1 -- 172.21.15.122:0/557417935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c107be0 msgr2=0x7f9a4c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a51c87700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c107be0 0x7f9a4c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a51c87700 1 -- 172.21.15.122:0/557417935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a4c102540 msgr2=0x7f9a4c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a51c87700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a4c102540 0x7f9a4c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a51c87700 1 -- 172.21.15.122:0/557417935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c19a0a0 con 0x7f9a4c101b80 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.344+0000 7f9a51c87700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a4c101b80 0x7f9a4c19f780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800b530 tx=0x7f9a4800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.345+0000 7f9a42ffd700 1 -- 172.21.15.122:0/557417935 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48015020 con 0x7f9a4c101b80 2024-05-03T17:54:35.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.345+0000 7f9a42ffd700 1 -- 172.21.15.122:0/557417935 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a48011ce0 con 0x7f9a4c101b80 2024-05-03T17:54:35.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.345+0000 7f9a53eeb700 1 -- 172.21.15.122:0/557417935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c19a390 con 0x7f9a4c101b80 2024-05-03T17:54:35.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.345+0000 7f9a42ffd700 1 -- 172.21.15.122:0/557417935 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48010e40 con 0x7f9a4c101b80 2024-05-03T17:54:35.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.345+0000 7f9a53eeb700 1 -- 172.21.15.122:0/557417935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c1a6650 con 0x7f9a4c101b80 2024-05-03T17:54:35.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.346+0000 7f9a53eeb700 1 -- 172.21.15.122:0/557417935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4c10a070 con 0x7f9a4c101b80 2024-05-03T17:54:35.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.347+0000 7f9a42ffd700 1 -- 172.21.15.122:0/557417935 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f9a48018020 con 0x7f9a4c101b80 2024-05-03T17:54:35.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.347+0000 7f9a42ffd700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a380b2990 0x7f9a380b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:35.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.348+0000 7f9a51486700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a380b2990 0x7f9a380b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:35.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.348+0000 7f9a42ffd700 1 -- 172.21.15.122:0/557417935 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f9a480e1f00 con 0x7f9a4c101b80 2024-05-03T17:54:35.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.348+0000 7f9a51486700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a380b2990 0x7f9a380b4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c1071f0 tx=0x7f9a3c00af60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.351+0000 7f9a42ffd700 1 -- 172.21.15.122:0/557417935 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9a4801f020 con 0x7f9a4c101b80 2024-05-03T17:54:35.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.566+0000 7f9a53eeb700 1 -- 172.21.15.122:0/557417935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9a4c19ae70 con 0x7f9a4c101b80 2024-05-03T17:54:35.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.567+0000 7f9a42ffd700 1 -- 172.21.15.122:0/557417935 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9a480a9cd0 con 0x7f9a4c101b80 2024-05-03T17:54:35.562 INFO:teuthology.orchestra.run.smithi122.stdout:107374182482 2024-05-03T17:54:35.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.569+0000 7f9a53eeb700 1 -- 172.21.15.122:0/557417935 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a380b2990 msgr2=0x7f9a380b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:35.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.569+0000 7f9a53eeb700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a380b2990 0x7f9a380b4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c1071f0 tx=0x7f9a3c00af60 comp rx=0 tx=0).stop 2024-05-03T17:54:35.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.569+0000 7f9a53eeb700 1 -- 172.21.15.122:0/557417935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a4c101b80 msgr2=0x7f9a4c19f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:35.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.569+0000 7f9a53eeb700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a4c101b80 0x7f9a4c19f780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800b530 tx=0x7f9a4800b840 comp rx=0 tx=0).stop 2024-05-03T17:54:35.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.569+0000 7f9a53eeb700 1 -- 172.21.15.122:0/557417935 shutdown_connections 2024-05-03T17:54:35.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.569+0000 7f9a53eeb700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9a380b2990 0x7f9a380b4e50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:35.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.569+0000 7f9a53eeb700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c107be0 0x7f9a4c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:35.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.569+0000 7f9a53eeb700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a4c101b80 0x7f9a4c19f780 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:35.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.569+0000 7f9a53eeb700 1 --2- 172.21.15.122:0/557417935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a4c102540 0x7f9a4c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:35.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.569+0000 7f9a53eeb700 1 -- 172.21.15.122:0/557417935 >> 172.21.15.122:0/557417935 conn(0x7f9a4c078a10 msgr2=0x7f9a4c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:35.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.570+0000 7f9a53eeb700 1 -- 172.21.15.122:0/557417935 shutdown_connections 2024-05-03T17:54:35.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:35.570+0000 7f9a53eeb700 1 -- 172.21.15.122:0/557417935 wait complete. 2024-05-03T17:54:36.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:35 smithi149 ceph-mon[33409]: pgmap v347: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-03T17:54:36.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:35 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:35 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:35 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:35 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:35 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/557417935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T17:54:36.060 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:35 smithi122 ceph-mon[29744]: pgmap v347: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-03T17:54:36.060 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:35 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.060 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:35 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.060 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:35 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.060 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:35 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.061 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:35 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/557417935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T17:54:36.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:35 smithi192 ceph-mon[33753]: pgmap v347: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-03T17:54:36.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:35 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:35 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:35 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:35 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:36.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:35 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/557417935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T17:54:36.347 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906984 got 274877906989 for osd.10 2024-05-03T17:54:36.347 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:36.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.595+0000 7fa3110c9700 1 -- 172.21.15.122:0/1193590629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa30c1076c0 msgr2=0x7fa30c10f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.595+0000 7fa3110c9700 1 --2- 172.21.15.122:0/1193590629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa30c1076c0 0x7fa30c10f290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa30400b0d0 tx=0x7fa30400b3e0 comp rx=0 tx=0).stop 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.595+0000 7fa3110c9700 1 -- 172.21.15.122:0/1193590629 shutdown_connections 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.595+0000 7fa3110c9700 1 --2- 172.21.15.122:0/1193590629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa30c1076c0 0x7fa30c10f290 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.595+0000 7fa3110c9700 1 --2- 172.21.15.122:0/1193590629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa30c102630 0x7fa30c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.595+0000 7fa3110c9700 1 --2- 172.21.15.122:0/1193590629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c102fe0 0x7fa30c106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.595+0000 7fa3110c9700 1 -- 172.21.15.122:0/1193590629 >> 172.21.15.122:0/1193590629 conn(0x7fa30c079350 msgr2=0x7fa30c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.595+0000 7fa3110c9700 1 -- 172.21.15.122:0/1193590629 shutdown_connections 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.595+0000 7fa3110c9700 1 -- 172.21.15.122:0/1193590629 wait complete. 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.596+0000 7fa3110c9700 1 Processor -- start 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.596+0000 7fa3110c9700 1 -- start start 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.596+0000 7fa3110c9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa30c102630 0x7fa30c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa3110c9700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa30c102fe0 0x7fa30c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa3110c9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c1076c0 0x7fa30c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:36.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa3110c9700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c1125d0 con 0x7fa30c102fe0 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa3110c9700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c112450 con 0x7fa30c1076c0 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa3110c9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c112750 con 0x7fa30c102630 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa30a59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa30c102fe0 0x7fa30c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa30a59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa30c102fe0 0x7fa30c19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49270/0 (socket says 172.21.15.122:49270) 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa30a59c700 1 -- 172.21.15.122:0/988782096 learned_addr learned my addr 172.21.15.122:0/988782096 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa30b59e700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c1076c0 0x7fa30c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa30ad9d700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa30c102630 0x7fa30c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa30a59c700 1 -- 172.21.15.122:0/988782096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa30c102630 msgr2=0x7fa30c19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa30a59c700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa30c102630 0x7fa30c19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa30a59c700 1 -- 172.21.15.122:0/988782096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c1076c0 msgr2=0x7fa30c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa30a59c700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c1076c0 0x7fa30c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:36.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa30a59c700 1 -- 172.21.15.122:0/988782096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c19a180 con 0x7fa30c102fe0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa30a59c700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa30c102fe0 0x7fa30c19fda0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa30000c8f0 tx=0x7fa30000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa2f3fff700 1 -- 172.21.15.122:0/988782096 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300007940 con 0x7fa30c102fe0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa3110c9700 1 -- 172.21.15.122:0/988782096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c19a470 con 0x7fa30c102fe0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa2f3fff700 1 -- 172.21.15.122:0/988782096 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa30000f450 con 0x7fa30c102fe0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa2f3fff700 1 -- 172.21.15.122:0/988782096 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300018660 con 0x7fa30c102fe0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.597+0000 7fa3110c9700 1 -- 172.21.15.122:0/988782096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c1a6730 con 0x7fa30c102fe0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.598+0000 7fa3110c9700 1 -- 172.21.15.122:0/988782096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa30c06a190 con 0x7fa30c102fe0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.600+0000 7fa2f3fff700 1 -- 172.21.15.122:0/988782096 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa30000f890 con 0x7fa30c102fe0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.600+0000 7fa2f3fff700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa2f40b2910 0x7fa2f40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.600+0000 7fa30ad9d700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa2f40b2910 0x7fa2f40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.600+0000 7fa2f3fff700 1 -- 172.21.15.122:0/988782096 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fa3000e2230 con 0x7fa30c102fe0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.601+0000 7fa30ad9d700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa2f40b2910 0x7fa2f40b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006fd0 tx=0x7fa2fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:36.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.605+0000 7fa2f3fff700 1 -- 172.21.15.122:0/988782096 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa300014030 con 0x7fa30c102fe0 2024-05-03T17:54:36.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.812+0000 7fa3110c9700 1 -- 172.21.15.122:0/988782096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fa30c02e510 con 0x7fa30c102fe0 2024-05-03T17:54:36.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.812+0000 7fa2f3fff700 1 -- 172.21.15.122:0/988782096 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa3000a9f80 con 0x7fa30c102fe0 2024-05-03T17:54:36.807 INFO:teuthology.orchestra.run.smithi122.stdout:253403070516 2024-05-03T17:54:36.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 -- 172.21.15.122:0/988782096 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa2f40b2910 msgr2=0x7fa2f40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:36.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa2f40b2910 0x7fa2f40b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006fd0 tx=0x7fa2fc008040 comp rx=0 tx=0).stop 2024-05-03T17:54:36.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 -- 172.21.15.122:0/988782096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa30c102fe0 msgr2=0x7fa30c19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:36.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa30c102fe0 0x7fa30c19fda0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa30000c8f0 tx=0x7fa30000cc00 comp rx=0 tx=0).stop 2024-05-03T17:54:36.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 -- 172.21.15.122:0/988782096 shutdown_connections 2024-05-03T17:54:36.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa2f40b2910 0x7fa2f40b4dd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:36.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c1076c0 0x7fa30c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:36.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa30c102630 0x7fa30c19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:36.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 --2- 172.21.15.122:0/988782096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa30c102fe0 0x7fa30c19fda0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:36.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 -- 172.21.15.122:0/988782096 >> 172.21.15.122:0/988782096 conn(0x7fa30c079350 msgr2=0x7fa30c06bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:36.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 -- 172.21.15.122:0/988782096 shutdown_connections 2024-05-03T17:54:36.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:36.814+0000 7fa3110c9700 1 -- 172.21.15.122:0/988782096 wait complete. 2024-05-03T17:54:37.885 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182476 got 107374182482 for osd.3 2024-05-03T17:54:37.886 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:37.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:37 smithi122 ceph-mon[29744]: pgmap v348: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:37.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:37 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/988782096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T17:54:38.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:37 smithi149 ceph-mon[33409]: pgmap v348: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:38.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:37 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/988782096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T17:54:38.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:37 smithi192 ceph-mon[33753]: pgmap v348: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:38.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:37 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/988782096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T17:54:38.323 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070510 got 253403070516 for osd.9 2024-05-03T17:54:38.323 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:38.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.515+0000 7fecf0169700 1 -- 172.21.15.122:0/1985709622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fece8102390 msgr2=0x7fece8102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:38.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.515+0000 7fecf0169700 1 --2- 172.21.15.122:0/1985709622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fece8102390 0x7fece8102810 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009990 tx=0x7fecd8009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:38.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.515+0000 7fecf0169700 1 -- 172.21.15.122:0/1985709622 shutdown_connections 2024-05-03T17:54:38.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.515+0000 7fecf0169700 1 --2- 172.21.15.122:0/1985709622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece8102d50 0x7fece810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:38.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.515+0000 7fecf0169700 1 --2- 172.21.15.122:0/1985709622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fece8108400 0x7fece81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:38.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.515+0000 7fecf0169700 1 --2- 172.21.15.122:0/1985709622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fece8102390 0x7fece8102810 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:38.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.515+0000 7fecf0169700 1 -- 172.21.15.122:0/1985709622 >> 172.21.15.122:0/1985709622 conn(0x7fece8079310 msgr2=0x7fece8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:38.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.515+0000 7fecf0169700 1 -- 172.21.15.122:0/1985709622 shutdown_connections 2024-05-03T17:54:38.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.515+0000 7fecf0169700 1 -- 172.21.15.122:0/1985709622 wait complete. 2024-05-03T17:54:38.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecf0169700 1 Processor -- start 2024-05-03T17:54:38.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecf0169700 1 -- start start 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecf0169700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fece8102390 0x7fece819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecf0169700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece8102d50 0x7fece819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecf0169700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fece8108400 0x7fece81a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecf0169700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fece81125a0 con 0x7fece8108400 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecf0169700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fece8112420 con 0x7fece8102d50 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecf0169700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fece8112720 con 0x7fece8102390 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecee706700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fece8108400 0x7fece81a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecee706700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fece8108400 0x7fece81a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44138/0 (socket says 172.21.15.122:44138) 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecee706700 1 -- 172.21.15.122:0/3018136003 learned_addr learned my addr 172.21.15.122:0/3018136003 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecedf05700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fece8102390 0x7fece819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7feced704700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece8102d50 0x7fece819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecee706700 1 -- 172.21.15.122:0/3018136003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fece8102390 msgr2=0x7fece819a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecee706700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fece8102390 0x7fece819a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:38.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecee706700 1 -- 172.21.15.122:0/3018136003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece8102d50 msgr2=0x7fece819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:38.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecee706700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece8102d50 0x7fece819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:38.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.516+0000 7fecee706700 1 -- 172.21.15.122:0/3018136003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece81a5f20 con 0x7fece8108400 2024-05-03T17:54:38.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.517+0000 7fecee706700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fece8108400 0x7fece81a59e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fece000b970 tx=0x7fece000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:38.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.517+0000 7fecdeffd700 1 -- 172.21.15.122:0/3018136003 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0019070 con 0x7fece8108400 2024-05-03T17:54:38.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.517+0000 7fecdeffd700 1 -- 172.21.15.122:0/3018136003 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece000cde0 con 0x7fece8108400 2024-05-03T17:54:38.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.517+0000 7fecf0169700 1 -- 172.21.15.122:0/3018136003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece81a6150 con 0x7fece8108400 2024-05-03T17:54:38.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.517+0000 7fecdeffd700 1 -- 172.21.15.122:0/3018136003 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0007370 con 0x7fece8108400 2024-05-03T17:54:38.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.517+0000 7fecf0169700 1 -- 172.21.15.122:0/3018136003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece81a6610 con 0x7fece8108400 2024-05-03T17:54:38.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.518+0000 7fecf0169700 1 -- 172.21.15.122:0/3018136003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece806a190 con 0x7fece8108400 2024-05-03T17:54:38.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.519+0000 7fecdeffd700 1 -- 172.21.15.122:0/3018136003 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fece001c070 con 0x7fece8108400 2024-05-03T17:54:38.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.520+0000 7fecdeffd700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fecd40b2910 0x7fecd40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:38.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.520+0000 7fecedf05700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fecd40b2910 0x7fecd40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:38.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.520+0000 7fecdeffd700 1 -- 172.21.15.122:0/3018136003 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fece00e1dd0 con 0x7fece8108400 2024-05-03T17:54:38.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.520+0000 7fecedf05700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fecd40b2910 0x7fecd40b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fece400ac50 tx=0x7fece400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:38.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.523+0000 7fecdeffd700 1 -- 172.21.15.122:0/3018136003 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fece0023070 con 0x7fece8108400 2024-05-03T17:54:38.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.733+0000 7fecf0169700 1 -- 172.21.15.122:0/3018136003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fece802e510 con 0x7fece8108400 2024-05-03T17:54:38.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.734+0000 7fecdeffd700 1 -- 172.21.15.122:0/3018136003 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fece00a9ba0 con 0x7fece8108400 2024-05-03T17:54:38.729 INFO:teuthology.orchestra.run.smithi122.stdout:34359738465 2024-05-03T17:54:38.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.736+0000 7fecf0169700 1 -- 172.21.15.122:0/3018136003 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fecd40b2910 msgr2=0x7fecd40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:38.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.736+0000 7fecf0169700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fecd40b2910 0x7fecd40b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fece400ac50 tx=0x7fece400a380 comp rx=0 tx=0).stop 2024-05-03T17:54:38.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.736+0000 7fecf0169700 1 -- 172.21.15.122:0/3018136003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fece8108400 msgr2=0x7fece81a59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:38.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.736+0000 7fecf0169700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fece8108400 0x7fece81a59e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fece000b970 tx=0x7fece000bc80 comp rx=0 tx=0).stop 2024-05-03T17:54:38.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.736+0000 7fecf0169700 1 -- 172.21.15.122:0/3018136003 shutdown_connections 2024-05-03T17:54:38.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.736+0000 7fecf0169700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fecd40b2910 0x7fecd40b4dd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:38.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.736+0000 7fecf0169700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fece8108400 0x7fece81a59e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:38.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.736+0000 7fecf0169700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fece8102390 0x7fece819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:38.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.736+0000 7fecf0169700 1 --2- 172.21.15.122:0/3018136003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece8102d50 0x7fece819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:38.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.736+0000 7fecf0169700 1 -- 172.21.15.122:0/3018136003 >> 172.21.15.122:0/3018136003 conn(0x7fece8079310 msgr2=0x7fece8100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:38.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.737+0000 7fecf0169700 1 -- 172.21.15.122:0/3018136003 shutdown_connections 2024-05-03T17:54:38.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:38.737+0000 7fecf0169700 1 -- 172.21.15.122:0/3018136003 wait complete. 2024-05-03T17:54:39.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.595+0000 7fed4286f700 1 -- 172.21.15.122:0/2894959379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed3c1076c0 msgr2=0x7fed3c10f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:39.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.595+0000 7fed4286f700 1 --2- 172.21.15.122:0/2894959379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed3c1076c0 0x7fed3c10f290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b0d0 tx=0x7fed3400b3e0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.595+0000 7fed4286f700 1 -- 172.21.15.122:0/2894959379 shutdown_connections 2024-05-03T17:54:39.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.595+0000 7fed4286f700 1 --2- 172.21.15.122:0/2894959379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed3c1076c0 0x7fed3c10f290 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.595+0000 7fed4286f700 1 --2- 172.21.15.122:0/2894959379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3c102630 0x7fed3c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.595+0000 7fed4286f700 1 --2- 172.21.15.122:0/2894959379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed3c102fe0 0x7fed3c106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.595+0000 7fed4286f700 1 -- 172.21.15.122:0/2894959379 >> 172.21.15.122:0/2894959379 conn(0x7fed3c079370 msgr2=0x7fed3c079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:39.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.595+0000 7fed4286f700 1 -- 172.21.15.122:0/2894959379 shutdown_connections 2024-05-03T17:54:39.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.595+0000 7fed4286f700 1 -- 172.21.15.122:0/2894959379 wait complete. 2024-05-03T17:54:39.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.595+0000 7fed4286f700 1 Processor -- start 2024-05-03T17:54:39.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.595+0000 7fed4286f700 1 -- start start 2024-05-03T17:54:39.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed4286f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed3c102630 0x7fed3c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed4286f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed3c102fe0 0x7fed3c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed4286f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3c1076c0 0x7fed3c1a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed4286f700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c112580 con 0x7fed3c102630 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed4286f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c112400 con 0x7fed3c102fe0 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed4286f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c112700 con 0x7fed3c1076c0 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed3c102630 0x7fed3c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed3c102630 0x7fed3c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44150/0 (socket says 172.21.15.122:44150) 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed3bfff700 1 -- 172.21.15.122:0/3816500168 learned_addr learned my addr 172.21.15.122:0/3816500168 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed3b7fe700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed3c102fe0 0x7fed3c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed40e0c700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3c1076c0 0x7fed3c1a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed3bfff700 1 -- 172.21.15.122:0/3816500168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3c1076c0 msgr2=0x7fed3c1a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed3bfff700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3c1076c0 0x7fed3c1a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed3bfff700 1 -- 172.21.15.122:0/3816500168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed3c102fe0 msgr2=0x7fed3c19af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:39.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed3bfff700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed3c102fe0 0x7fed3c19af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed3bfff700 1 -- 172.21.15.122:0/3816500168 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c1a5fa0 con 0x7fed3c102630 2024-05-03T17:54:39.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed40e0c700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3c1076c0 0x7fed3c1a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:54:39.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed3bfff700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed3c102630 0x7fed3c19a9e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00e9c0 tx=0x7fed2c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:39.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.596+0000 7fed397fa700 1 -- 172.21.15.122:0/3816500168 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c00cbd0 con 0x7fed3c102630 2024-05-03T17:54:39.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.597+0000 7fed4286f700 1 -- 172.21.15.122:0/3816500168 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c1a61d0 con 0x7fed3c102630 2024-05-03T17:54:39.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.597+0000 7fed397fa700 1 -- 172.21.15.122:0/3816500168 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed2c00cd70 con 0x7fed3c102630 2024-05-03T17:54:39.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.597+0000 7fed397fa700 1 -- 172.21.15.122:0/3816500168 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c005610 con 0x7fed3c102630 2024-05-03T17:54:39.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.597+0000 7fed4286f700 1 -- 172.21.15.122:0/3816500168 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c1a66e0 con 0x7fed3c102630 2024-05-03T17:54:39.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.597+0000 7fed4286f700 1 -- 172.21.15.122:0/3816500168 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed3c06a190 con 0x7fed3c102630 2024-05-03T17:54:39.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.599+0000 7fed397fa700 1 -- 172.21.15.122:0/3816500168 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fed2c004750 con 0x7fed3c102630 2024-05-03T17:54:39.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.599+0000 7fed397fa700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fed240b2910 0x7fed240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:39.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.600+0000 7fed3b7fe700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fed240b2910 0x7fed240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:39.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.600+0000 7fed397fa700 1 -- 172.21.15.122:0/3816500168 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fed2c0e27d0 con 0x7fed3c102630 2024-05-03T17:54:39.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.600+0000 7fed3b7fe700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fed240b2910 0x7fed240b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed3c101f20 tx=0x7fed30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:39.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.604+0000 7fed397fa700 1 -- 172.21.15.122:0/3816500168 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fed2c054d70 con 0x7fed3c102630 2024-05-03T17:54:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.816+0000 7fed4286f700 1 -- 172.21.15.122:0/3816500168 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fed3c02e510 con 0x7fed3c102630 2024-05-03T17:54:39.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.816+0000 7fed397fa700 1 -- 172.21.15.122:0/3816500168 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fed2c0aa490 con 0x7fed3c102630 2024-05-03T17:54:39.811 INFO:teuthology.orchestra.run.smithi122.stdout:133143986253 2024-05-03T17:54:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 -- 172.21.15.122:0/3816500168 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fed240b2910 msgr2=0x7fed240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fed240b2910 0x7fed240b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed3c101f20 tx=0x7fed30009260 comp rx=0 tx=0).stop 2024-05-03T17:54:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 -- 172.21.15.122:0/3816500168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed3c102630 msgr2=0x7fed3c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed3c102630 0x7fed3c19a9e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00e9c0 tx=0x7fed2c00ecd0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 -- 172.21.15.122:0/3816500168 shutdown_connections 2024-05-03T17:54:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fed240b2910 0x7fed240b4dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3c1076c0 0x7fed3c1a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed3c102630 0x7fed3c19a9e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 --2- 172.21.15.122:0/3816500168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed3c102fe0 0x7fed3c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 -- 172.21.15.122:0/3816500168 >> 172.21.15.122:0/3816500168 conn(0x7fed3c079370 msgr2=0x7fed3c1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 -- 172.21.15.122:0/3816500168 shutdown_connections 2024-05-03T17:54:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7fed4286f700 1 -- 172.21.15.122:0/3816500168 wait complete. 2024-05-03T17:54:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.813+0000 7f1aa75ba700 1 -- 172.21.15.122:0/2954825599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1aa0075710 msgr2=0x7f1aa0075af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.813+0000 7f1aa75ba700 1 --2- 172.21.15.122:0/2954825599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1aa0075710 0x7f1aa0075af0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009990 tx=0x7f1a9c009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.813+0000 7f1aa75ba700 1 -- 172.21.15.122:0/2954825599 shutdown_connections 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.813+0000 7f1aa75ba700 1 --2- 172.21.15.122:0/2954825599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1aa010fea0 0x7f1aa0112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.813+0000 7f1aa75ba700 1 --2- 172.21.15.122:0/2954825599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1aa0075710 0x7f1aa0075af0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.813+0000 7f1aa75ba700 1 --2- 172.21.15.122:0/2954825599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa0076030 0x7f1aa010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.813+0000 7f1aa75ba700 1 -- 172.21.15.122:0/2954825599 >> 172.21.15.122:0/2954825599 conn(0x7f1aa007a690 msgr2=0x7f1aa007aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.814+0000 7f1aa75ba700 1 -- 172.21.15.122:0/2954825599 shutdown_connections 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.814+0000 7f1aa75ba700 1 -- 172.21.15.122:0/2954825599 wait complete. 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.814+0000 7f1aa75ba700 1 Processor -- start 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.814+0000 7f1aa75ba700 1 -- start start 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa75ba700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1aa0075710 0x7f1aa019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa75ba700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa0076030 0x7f1aa019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa75ba700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1aa010fea0 0x7f1aa01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa75ba700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa01158c0 con 0x7f1aa010fea0 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa75ba700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa0115740 con 0x7f1aa0075710 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa75ba700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa0115a40 con 0x7f1aa0076030 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa5b57700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1aa010fea0 0x7f1aa01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa5b57700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1aa010fea0 0x7f1aa01a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44162/0 (socket says 172.21.15.122:44162) 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa5b57700 1 -- 172.21.15.122:0/1444916982 learned_addr learned my addr 172.21.15.122:0/1444916982 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa4b55700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa0076030 0x7f1aa019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa5356700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1aa0075710 0x7f1aa019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa5b57700 1 -- 172.21.15.122:0/1444916982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa0076030 msgr2=0x7f1aa019c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa5b57700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa0076030 0x7f1aa019c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa5b57700 1 -- 172.21.15.122:0/1444916982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1aa0075710 msgr2=0x7f1aa019bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa5b57700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1aa0075710 0x7f1aa019bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa5b57700 1 -- 172.21.15.122:0/1444916982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa01a0d90 con 0x7f1aa010fea0 2024-05-03T17:54:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa5b57700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1aa010fea0 0x7f1aa01a0650 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a98007d70 tx=0x7f1a9800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:39.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1a967fc700 1 -- 172.21.15.122:0/1444916982 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98016020 con 0x7f1aa010fea0 2024-05-03T17:54:39.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa75ba700 1 -- 172.21.15.122:0/1444916982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa01a1080 con 0x7f1aa010fea0 2024-05-03T17:54:39.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1a967fc700 1 -- 172.21.15.122:0/1444916982 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9800ad40 con 0x7f1aa010fea0 2024-05-03T17:54:39.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1a967fc700 1 -- 172.21.15.122:0/1444916982 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a980115e0 con 0x7f1aa010fea0 2024-05-03T17:54:39.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.815+0000 7f1aa75ba700 1 -- 172.21.15.122:0/1444916982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa01a8900 con 0x7f1aa010fea0 2024-05-03T17:54:39.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.816+0000 7f1aa75ba700 1 -- 172.21.15.122:0/1444916982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa006a190 con 0x7f1aa010fea0 2024-05-03T17:54:39.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.818+0000 7f1a967fc700 1 -- 172.21.15.122:0/1444916982 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f1a98019020 con 0x7f1aa010fea0 2024-05-03T17:54:39.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.818+0000 7f1a967fc700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1a8c0b2910 0x7f1a8c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:39.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7f1aa5356700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1a8c0b2910 0x7f1a8c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:39.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7f1a967fc700 1 -- 172.21.15.122:0/1444916982 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f1a980e2630 con 0x7f1aa010fea0 2024-05-03T17:54:39.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.819+0000 7f1aa5356700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1a8c0b2910 0x7f1a8c0b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c006010 tx=0x7f1a9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:39.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:39.825+0000 7f1a967fc700 1 -- 172.21.15.122:0/1444916982 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1a98020020 con 0x7f1aa010fea0 2024-05-03T17:54:39.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:39 smithi122 ceph-mon[29744]: pgmap v349: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:39.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:39 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3018136003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T17:54:40.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.038+0000 7f1aa75ba700 1 -- 172.21.15.122:0/1444916982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1aa019d0e0 con 0x7f1aa010fea0 2024-05-03T17:54:40.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.038+0000 7f1a967fc700 1 -- 172.21.15.122:0/1444916982 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1a980aa400 con 0x7f1aa010fea0 2024-05-03T17:54:40.033 INFO:teuthology.orchestra.run.smithi122.stdout:201863462974 2024-05-03T17:54:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.040+0000 7f1aa75ba700 1 -- 172.21.15.122:0/1444916982 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1a8c0b2910 msgr2=0x7f1a8c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.040+0000 7f1aa75ba700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1a8c0b2910 0x7f1a8c0b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c006010 tx=0x7f1a9c01a040 comp rx=0 tx=0).stop 2024-05-03T17:54:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.040+0000 7f1aa75ba700 1 -- 172.21.15.122:0/1444916982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1aa010fea0 msgr2=0x7f1aa01a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.040+0000 7f1aa75ba700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1aa010fea0 0x7f1aa01a0650 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a98007d70 tx=0x7f1a9800a340 comp rx=0 tx=0).stop 2024-05-03T17:54:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.041+0000 7f1aa75ba700 1 -- 172.21.15.122:0/1444916982 shutdown_connections 2024-05-03T17:54:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.041+0000 7f1aa75ba700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1a8c0b2910 0x7f1a8c0b4dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.041+0000 7f1aa75ba700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1aa010fea0 0x7f1aa01a0650 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.041+0000 7f1aa75ba700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1aa0075710 0x7f1aa019bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.041+0000 7f1aa75ba700 1 --2- 172.21.15.122:0/1444916982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa0076030 0x7f1aa019c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.041+0000 7f1aa75ba700 1 -- 172.21.15.122:0/1444916982 >> 172.21.15.122:0/1444916982 conn(0x7f1aa007a690 msgr2=0x7f1aa01006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.041+0000 7f1aa75ba700 1 -- 172.21.15.122:0/1444916982 shutdown_connections 2024-05-03T17:54:40.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:40.041+0000 7f1aa75ba700 1 -- 172.21.15.122:0/1444916982 wait complete. 2024-05-03T17:54:40.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:39 smithi149 ceph-mon[33409]: pgmap v349: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:40.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:39 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3018136003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T17:54:40.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:39 smithi192 ceph-mon[33753]: pgmap v349: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:40.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:39 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3018136003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T17:54:40.247 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574935 got 55834574938 for osd.1 2024-05-03T17:54:40.247 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:40.861 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738459 got 34359738465 for osd.0 2024-05-03T17:54:40.861 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:41.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:40 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3816500168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T17:54:41.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:40 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/1444916982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T17:54:41.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:40 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3816500168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T17:54:41.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:40 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/1444916982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T17:54:41.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:40 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3816500168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T17:54:41.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:40 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1444916982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T17:54:42.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:41 smithi149 ceph-mon[33409]: pgmap v350: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:42.053 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:41 smithi122 ceph-mon[29744]: pgmap v350: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:42.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:41 smithi192 ceph-mon[33753]: pgmap v350: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:43.003 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986247 got 133143986253 for osd.4 2024-05-03T17:54:43.003 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.194+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/579344683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab4103040 msgr2=0x7f8ab4107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.194+0000 7f8ab8ac5700 1 --2- 172.21.15.122:0/579344683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab4103040 0x7f8ab4107000 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80099c0 tx=0x7f8aa8009cd0 comp rx=0 tx=0).stop 2024-05-03T17:54:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.194+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/579344683 shutdown_connections 2024-05-03T17:54:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.194+0000 7f8ab8ac5700 1 --2- 172.21.15.122:0/579344683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab41076e0 0x7f8ab410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.194+0000 7f8ab8ac5700 1 --2- 172.21.15.122:0/579344683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ab4102690 0x7f8ab4102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.194+0000 7f8ab8ac5700 1 --2- 172.21.15.122:0/579344683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab4103040 0x7f8ab4107000 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.194+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/579344683 >> 172.21.15.122:0/579344683 conn(0x7f8ab40783f0 msgr2=0x7f8ab4078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.194+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/579344683 shutdown_connections 2024-05-03T17:54:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.194+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/579344683 wait complete. 2024-05-03T17:54:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.194+0000 7f8ab8ac5700 1 Processor -- start 2024-05-03T17:54:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.194+0000 7f8ab8ac5700 1 -- start start 2024-05-03T17:54:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab8ac5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ab4102690 0x7f8ab419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab8ac5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4103040 0x7f8ab419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab8ac5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab41076e0 0x7f8ab4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab8ac5700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4112640 con 0x7f8ab41076e0 2024-05-03T17:54:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab8ac5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41124c0 con 0x7f8ab4102690 2024-05-03T17:54:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab8ac5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41127c0 con 0x7f8ab4103040 2024-05-03T17:54:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab2d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab41076e0 0x7f8ab4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab2d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab41076e0 0x7f8ab4198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44176/0 (socket says 172.21.15.122:44176) 2024-05-03T17:54:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab2d9d700 1 -- 172.21.15.122:0/3325455411 learned_addr learned my addr 172.21.15.122:0/3325455411 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab1d9b700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4103040 0x7f8ab419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab259c700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ab4102690 0x7f8ab419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab2d9d700 1 -- 172.21.15.122:0/3325455411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4103040 msgr2=0x7f8ab419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab2d9d700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4103040 0x7f8ab419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab2d9d700 1 -- 172.21.15.122:0/3325455411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ab4102690 msgr2=0x7f8ab419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab2d9d700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ab4102690 0x7f8ab419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab2d9d700 1 -- 172.21.15.122:0/3325455411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab41990a0 con 0x7f8ab41076e0 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab1d9b700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4103040 0x7f8ab419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab259c700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ab4102690 0x7f8ab419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab2d9d700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab41076e0 0x7f8ab4198940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8aac007ee0 tx=0x7f8aac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8aa37fe700 1 -- 172.21.15.122:0/3325455411 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac00dbd0 con 0x7f8ab41076e0 2024-05-03T17:54:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/3325455411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab4199390 con 0x7f8ab41076e0 2024-05-03T17:54:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8aa37fe700 1 -- 172.21.15.122:0/3325455411 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aac00dd70 con 0x7f8ab41076e0 2024-05-03T17:54:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8aa37fe700 1 -- 172.21.15.122:0/3325455411 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac005990 con 0x7f8ab41076e0 2024-05-03T17:54:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.195+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/3325455411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab41a5650 con 0x7f8ab41076e0 2024-05-03T17:54:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.196+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/3325455411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab406a190 con 0x7f8ab41076e0 2024-05-03T17:54:43.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.198+0000 7f8aa37fe700 1 -- 172.21.15.122:0/3325455411 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8aac01a020 con 0x7f8ab41076e0 2024-05-03T17:54:43.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.198+0000 7f8aa37fe700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8a9c0b2910 0x7f8a9c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:43.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.198+0000 7f8ab259c700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8a9c0b2910 0x7f8a9c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:43.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.198+0000 7f8aa37fe700 1 -- 172.21.15.122:0/3325455411 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f8aac0e1ae0 con 0x7f8ab41076e0 2024-05-03T17:54:43.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.199+0000 7f8ab259c700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8a9c0b2910 0x7f8a9c0b4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4199a80 tx=0x7f8aa4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:43.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.203+0000 7f8aa37fe700 1 -- 172.21.15.122:0/3325455411 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8aac010070 con 0x7f8ab41076e0 2024-05-03T17:54:43.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.415+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/3325455411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f8ab402e510 con 0x7f8ab41076e0 2024-05-03T17:54:43.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.416+0000 7f8aa37fe700 1 -- 172.21.15.122:0/3325455411 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8aac0a97a0 con 0x7f8ab41076e0 2024-05-03T17:54:43.411 INFO:teuthology.orchestra.run.smithi122.stdout:296352743466 2024-05-03T17:54:43.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/3325455411 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8a9c0b2910 msgr2=0x7f8a9c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:43.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8a9c0b2910 0x7f8a9c0b4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4199a80 tx=0x7f8aa4008040 comp rx=0 tx=0).stop 2024-05-03T17:54:43.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/3325455411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab41076e0 msgr2=0x7f8ab4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:43.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab41076e0 0x7f8ab4198940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8aac007ee0 tx=0x7f8aac00d3b0 comp rx=0 tx=0).stop 2024-05-03T17:54:43.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/3325455411 shutdown_connections 2024-05-03T17:54:43.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8a9c0b2910 0x7f8a9c0b4dd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:43.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab41076e0 0x7f8ab4198940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:43.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ab4102690 0x7f8ab419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:43.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 --2- 172.21.15.122:0/3325455411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4103040 0x7f8ab419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:43.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/3325455411 >> 172.21.15.122:0/3325455411 conn(0x7f8ab40783f0 msgr2=0x7f8ab4100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:43.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/3325455411 shutdown_connections 2024-05-03T17:54:43.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:43.418+0000 7f8ab8ac5700 1 -- 172.21.15.122:0/3325455411 wait complete. 2024-05-03T17:54:43.652 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462968 got 201863462974 for osd.7 2024-05-03T17:54:43.652 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:43.884 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:43 smithi122 ceph-mon[29744]: pgmap v351: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:43.884 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:43 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3325455411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T17:54:44.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:43 smithi149 ceph-mon[33409]: pgmap v351: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:44.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:43 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3325455411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T17:54:44.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:43 smithi192 ceph-mon[33753]: pgmap v351: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:44.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:43 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3325455411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T17:54:44.475 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743461 got 296352743466 for osd.11 2024-05-03T17:54:44.475 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:44.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.755+0000 7f3533069700 1 -- 172.21.15.122:0/3529868544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c103750 msgr2=0x7f352c103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:44.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.755+0000 7f3533069700 1 --2- 172.21.15.122:0/3529868544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c103750 0x7f352c103b30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f35200099c0 tx=0x7f3520009cd0 comp rx=0 tx=0).stop 2024-05-03T17:54:44.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.755+0000 7f3533069700 1 -- 172.21.15.122:0/3529868544 shutdown_connections 2024-05-03T17:54:44.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.755+0000 7f3533069700 1 --2- 172.21.15.122:0/3529868544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f352c108850 0x7f352c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:44.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.755+0000 7f3533069700 1 --2- 172.21.15.122:0/3529868544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c103750 0x7f352c103b30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:44.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.755+0000 7f3533069700 1 --2- 172.21.15.122:0/3529868544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c104100 0x7f352c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:44.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.755+0000 7f3533069700 1 -- 172.21.15.122:0/3529868544 >> 172.21.15.122:0/3529868544 conn(0x7f352c07a600 msgr2=0x7f352c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:44.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.755+0000 7f3533069700 1 -- 172.21.15.122:0/3529868544 shutdown_connections 2024-05-03T17:54:44.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.755+0000 7f3533069700 1 -- 172.21.15.122:0/3529868544 wait complete. 2024-05-03T17:54:44.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.756+0000 7f3533069700 1 Processor -- start 2024-05-03T17:54:44.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.756+0000 7f3533069700 1 -- start start 2024-05-03T17:54:44.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.756+0000 7f3533069700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f352c103750 0x7f352c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.756+0000 7f3533069700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c104100 0x7f352c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.756+0000 7f3533069700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c108850 0x7f352c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.756+0000 7f3533069700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f352c113690 con 0x7f352c104100 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.756+0000 7f3533069700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f352c113510 con 0x7f352c103750 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.756+0000 7f3533069700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f352c113810 con 0x7f352c108850 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.756+0000 7f352bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c104100 0x7f352c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.756+0000 7f352bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c104100 0x7f352c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44200/0 (socket says 172.21.15.122:44200) 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.756+0000 7f352bfff700 1 -- 172.21.15.122:0/394222768 learned_addr learned my addr 172.21.15.122:0/394222768 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f3530e05700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f352c103750 0x7f352c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f352bfff700 1 -- 172.21.15.122:0/394222768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c108850 msgr2=0x7f352c1999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f352bfff700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c108850 0x7f352c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f352bfff700 1 -- 172.21.15.122:0/394222768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f352c103750 msgr2=0x7f352c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:44.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f352bfff700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f352c103750 0x7f352c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:44.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f352bfff700 1 -- 172.21.15.122:0/394222768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c19a130 con 0x7f352c104100 2024-05-03T17:54:44.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f352bfff700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c104100 0x7f352c19fd00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f351800b530 tx=0x7f351800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:44.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f3529ffb700 1 -- 172.21.15.122:0/394222768 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518015020 con 0x7f352c104100 2024-05-03T17:54:44.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f3529ffb700 1 -- 172.21.15.122:0/394222768 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3518011ce0 con 0x7f352c104100 2024-05-03T17:54:44.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f3533069700 1 -- 172.21.15.122:0/394222768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352c19a420 con 0x7f352c104100 2024-05-03T17:54:44.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f3529ffb700 1 -- 172.21.15.122:0/394222768 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518010e40 con 0x7f352c104100 2024-05-03T17:54:44.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.757+0000 7f3533069700 1 -- 172.21.15.122:0/394222768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352c1a6690 con 0x7f352c104100 2024-05-03T17:54:44.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.758+0000 7f3533069700 1 -- 172.21.15.122:0/394222768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352c10ba90 con 0x7f352c104100 2024-05-03T17:54:44.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.759+0000 7f3529ffb700 1 -- 172.21.15.122:0/394222768 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f3518018020 con 0x7f352c104100 2024-05-03T17:54:44.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.760+0000 7f3529ffb700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f351c0b2910 0x7f351c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:44.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.760+0000 7f3530e05700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f351c0b2910 0x7f351c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:44.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.760+0000 7f3529ffb700 1 -- 172.21.15.122:0/394222768 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f35180e19c0 con 0x7f352c104100 2024-05-03T17:54:44.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.760+0000 7f3530e05700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f351c0b2910 0x7f351c0b4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f352c108300 tx=0x7f3520003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:44.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.764+0000 7f3529ffb700 1 -- 172.21.15.122:0/394222768 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f351801f020 con 0x7f352c104100 2024-05-03T17:54:44.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.976+0000 7f3533069700 1 -- 172.21.15.122:0/394222768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f352c10b470 con 0x7f352c104100 2024-05-03T17:54:44.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.977+0000 7f3529ffb700 1 -- 172.21.15.122:0/394222768 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f35180a9790 con 0x7f352c104100 2024-05-03T17:54:44.972 INFO:teuthology.orchestra.run.smithi122.stdout:180388626500 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 -- 172.21.15.122:0/394222768 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f351c0b2910 msgr2=0x7f351c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f351c0b2910 0x7f351c0b4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f352c108300 tx=0x7f3520003680 comp rx=0 tx=0).stop 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 -- 172.21.15.122:0/394222768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c104100 msgr2=0x7f352c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c104100 0x7f352c19fd00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f351800b530 tx=0x7f351800b840 comp rx=0 tx=0).stop 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 -- 172.21.15.122:0/394222768 shutdown_connections 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f351c0b2910 0x7f351c0b4dd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c108850 0x7f352c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f352c103750 0x7f352c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 --2- 172.21.15.122:0/394222768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c104100 0x7f352c19fd00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 -- 172.21.15.122:0/394222768 >> 172.21.15.122:0/394222768 conn(0x7f352c07a600 msgr2=0x7f352c1099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 -- 172.21.15.122:0/394222768 shutdown_connections 2024-05-03T17:54:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:44.979+0000 7f3533069700 1 -- 172.21.15.122:0/394222768 wait complete. 2024-05-03T17:54:46.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:45 smithi149 ceph-mon[33409]: pgmap v352: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:46.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:45 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/394222768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T17:54:46.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:45 smithi192 ceph-mon[33753]: pgmap v352: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:46.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:45 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/394222768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T17:54:46.141 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:45 smithi122 ceph-mon[29744]: pgmap v352: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:46.141 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:45 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/394222768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T17:54:46.167 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626495 got 180388626500 for osd.6 2024-05-03T17:54:46.168 DEBUG:teuthology.parallel:result is None 2024-05-03T17:54:46.168 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-03T17:54:46.168 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T17:54:46.452 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:47.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:47 smithi122 ceph-mon[29744]: pgmap v353: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:48.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:47 smithi149 ceph-mon[33409]: pgmap v353: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:48.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:47 smithi192 ceph-mon[33753]: pgmap v353: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:48.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.220+0000 7f8ba5594700 1 -- 172.21.15.122:0/4131503598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba006b180 msgr2=0x7f8ba006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:48.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.220+0000 7f8ba5594700 1 --2- 172.21.15.122:0/4131503598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba006b180 0x7f8ba006b600 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009990 tx=0x7f8b94009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.220+0000 7f8ba5594700 1 -- 172.21.15.122:0/4131503598 shutdown_connections 2024-05-03T17:54:48.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.220+0000 7f8ba5594700 1 --2- 172.21.15.122:0/4131503598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba006bb40 0x7f8ba0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.220+0000 7f8ba5594700 1 --2- 172.21.15.122:0/4131503598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ba0109510 0x7f8ba01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.220+0000 7f8ba5594700 1 --2- 172.21.15.122:0/4131503598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba006b180 0x7f8ba006b600 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.220+0000 7f8ba5594700 1 -- 172.21.15.122:0/4131503598 >> 172.21.15.122:0/4131503598 conn(0x7f8ba0079350 msgr2=0x7f8ba0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:48.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.220+0000 7f8ba5594700 1 -- 172.21.15.122:0/4131503598 shutdown_connections 2024-05-03T17:54:48.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.220+0000 7f8ba5594700 1 -- 172.21.15.122:0/4131503598 wait complete. 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.220+0000 7f8ba5594700 1 Processor -- start 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.220+0000 7f8ba5594700 1 -- start start 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8ba5594700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ba006b180 0x7f8ba019be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8ba5594700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba006bb40 0x7f8ba019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8ba5594700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0109510 0x7f8ba01a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8ba5594700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0113640 con 0x7f8ba006bb40 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8ba5594700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01134c0 con 0x7f8ba0109510 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8ba5594700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01137c0 con 0x7f8ba006b180 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9e7fc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba006bb40 0x7f8ba019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9e7fc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba006bb40 0x7f8ba019c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44220/0 (socket says 172.21.15.122:44220) 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9e7fc700 1 -- 172.21.15.122:0/2206521806 learned_addr learned my addr 172.21.15.122:0/2206521806 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9effd700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ba006b180 0x7f8ba019be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:48.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9f7fe700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0109510 0x7f8ba01a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9e7fc700 1 -- 172.21.15.122:0/2206521806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ba006b180 msgr2=0x7f8ba019be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9e7fc700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ba006b180 0x7f8ba019be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9e7fc700 1 -- 172.21.15.122:0/2206521806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0109510 msgr2=0x7f8ba01a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9e7fc700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0109510 0x7f8ba01a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9e7fc700 1 -- 172.21.15.122:0/2206521806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba01a0e60 con 0x7f8ba006bb40 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9f7fe700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0109510 0x7f8ba01a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b9e7fc700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba006bb40 0x7f8ba019c390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8b94000c00 tx=0x7f8b940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b87fff700 1 -- 172.21.15.122:0/2206521806 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94027070 con 0x7f8ba006bb40 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8ba5594700 1 -- 172.21.15.122:0/2206521806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba01a10f0 con 0x7f8ba006bb40 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b87fff700 1 -- 172.21.15.122:0/2206521806 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b94003f20 con 0x7f8ba006bb40 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8b87fff700 1 -- 172.21.15.122:0/2206521806 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94022cf0 con 0x7f8ba006bb40 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.221+0000 7f8ba5594700 1 -- 172.21.15.122:0/2206521806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba01a89d0 con 0x7f8ba006bb40 2024-05-03T17:54:48.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.222+0000 7f8ba5594700 1 -- 172.21.15.122:0/2206521806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba006a190 con 0x7f8ba006bb40 2024-05-03T17:54:48.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.224+0000 7f8b87fff700 1 -- 172.21.15.122:0/2206521806 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8b9402a070 con 0x7f8ba006bb40 2024-05-03T17:54:48.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.224+0000 7f8b87fff700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8b880b2910 0x7f8b880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:48.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.224+0000 7f8b9effd700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8b880b2910 0x7f8b880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:48.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.224+0000 7f8b87fff700 1 -- 172.21.15.122:0/2206521806 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f8b940e33e0 con 0x7f8ba006bb40 2024-05-03T17:54:48.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.225+0000 7f8b9effd700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8b880b2910 0x7f8b880b4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8b900045b0 tx=0x7f8b9000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:48.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.228+0000 7f8b87fff700 1 -- 172.21.15.122:0/2206521806 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8b94013070 con 0x7f8ba006bb40 2024-05-03T17:54:48.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.449+0000 7f8ba5594700 1 -- 172.21.15.122:0/2206521806 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ba010ed90 con 0x7f8b880b2910 2024-05-03T17:54:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.456+0000 7f8b87fff700 1 -- 172.21.15.122:0/2206521806 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258870 (secure 0 0 0) 0x7f8ba010ed90 con 0x7f8b880b2910 2024-05-03T17:54:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:54:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 -- 172.21.15.122:0/2206521806 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8b880b2910 msgr2=0x7f8b880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8b880b2910 0x7f8b880b4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8b900045b0 tx=0x7f8b9000b3c0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 -- 172.21.15.122:0/2206521806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba006bb40 msgr2=0x7f8ba019c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba006bb40 0x7f8ba019c390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8b94000c00 tx=0x7f8b940047b0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 -- 172.21.15.122:0/2206521806 shutdown_connections 2024-05-03T17:54:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8b880b2910 0x7f8b880b4dd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0109510 0x7f8ba01a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ba006b180 0x7f8ba019be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 --2- 172.21.15.122:0/2206521806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba006bb40 0x7f8ba019c390 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 -- 172.21.15.122:0/2206521806 >> 172.21.15.122:0/2206521806 conn(0x7f8ba0079350 msgr2=0x7f8ba010e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 -- 172.21.15.122:0/2206521806 shutdown_connections 2024-05-03T17:54:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:48.459+0000 7f8ba5594700 1 -- 172.21.15.122:0/2206521806 wait complete. 2024-05-03T17:54:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-05-03T17:54:49.119 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-05-03T17:54:46.570087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649811,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":764,"num_read_kb":469,"num_write":289,"num_write_kb":2258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376344,"kb_used_data":10500,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565480,"statfs":{"total":1151940427776,"available":1151555051520,"internally_reserved":0,"allocated":10752000,"data_stored":6958113,"data_compressed":107130,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003195"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870216+0000","last_change":"2024-05-03T17:52:35.725057+0000","last_active":"2024-05-03T17:53:19.870216+0000","last_peered":"2024-05-03T17:53:19.870216+0000","last_clean":"2024-05-03T17:53:19.870216+0000","last_became_active":"2024-05-03T17:52:35.724632+0000","last_became_peered":"2024-05-03T17:52:35.724632+0000","last_unstale":"2024-05-03T17:53:19.870216+0000","last_undegraded":"2024-05-03T17:53:19.870216+0000","last_fullsized":"2024-05-03T17:53:19.870216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:46.065357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870713+0000","last_change":"2024-05-03T17:52:35.728500+0000","last_active":"2024-05-03T17:53:19.870713+0000","last_peered":"2024-05-03T17:53:19.870713+0000","last_clean":"2024-05-03T17:53:19.870713+0000","last_became_active":"2024-05-03T17:52:35.728210+0000","last_became_peered":"2024-05-03T17:52:35.728210+0000","last_unstale":"2024-05-03T17:53:19.870713+0000","last_undegraded":"2024-05-03T17:53:19.870713+0000","last_fullsized":"2024-05-03T17:53:19.870713+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:33:23.410617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870687+0000","last_change":"2024-05-03T17:52:35.726300+0000","last_active":"2024-05-03T17:53:19.870687+0000","last_peered":"2024-05-03T17:53:19.870687+0000","last_clean":"2024-05-03T17:53:19.870687+0000","last_became_active":"2024-05-03T17:52:35.725994+0000","last_became_peered":"2024-05-03T17:52:35.725994+0000","last_unstale":"2024-05-03T17:53:19.870687+0000","last_undegraded":"2024-05-03T17:53:19.870687+0000","last_fullsized":"2024-05-03T17:53:19.870687+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:31:19.723453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077329+0000","last_change":"2024-05-03T17:52:35.728105+0000","last_active":"2024-05-03T17:53:20.077329+0000","last_peered":"2024-05-03T17:53:20.077329+0000","last_clean":"2024-05-03T17:53:20.077329+0000","last_became_active":"2024-05-03T17:52:35.727762+0000","last_became_peered":"2024-05-03T17:52:35.727762+0000","last_unstale":"2024-05-03T17:53:20.077329+0000","last_undegraded":"2024-05-03T17:53:20.077329+0000","last_fullsized":"2024-05-03T17:53:20.077329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.640057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870597+0000","last_change":"2024-05-03T17:52:35.734516+0000","last_active":"2024-05-03T17:53:19.870597+0000","last_peered":"2024-05-03T17:53:19.870597+0000","last_clean":"2024-05-03T17:53:19.870597+0000","last_became_active":"2024-05-03T17:52:35.734125+0000","last_became_peered":"2024-05-03T17:52:35.734125+0000","last_unstale":"2024-05-03T17:53:19.870597+0000","last_undegraded":"2024-05-03T17:53:19.870597+0000","last_fullsized":"2024-05-03T17:53:19.870597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:39:25.103137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871539+0000","last_change":"2024-05-03T17:52:35.723427+0000","last_active":"2024-05-03T17:53:19.871539+0000","last_peered":"2024-05-03T17:53:19.871539+0000","last_clean":"2024-05-03T17:53:19.871539+0000","last_became_active":"2024-05-03T17:52:35.723113+0000","last_became_peered":"2024-05-03T17:52:35.723113+0000","last_unstale":"2024-05-03T17:53:19.871539+0000","last_undegraded":"2024-05-03T17:53:19.871539+0000","last_fullsized":"2024-05-03T17:53:19.871539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:37:14.606347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077441+0000","last_change":"2024-05-03T17:52:35.730858+0000","last_active":"2024-05-03T17:53:20.077441+0000","last_peered":"2024-05-03T17:53:20.077441+0000","last_clean":"2024-05-03T17:53:20.077441+0000","last_became_active":"2024-05-03T17:52:35.730453+0000","last_became_peered":"2024-05-03T17:52:35.730453+0000","last_unstale":"2024-05-03T17:53:20.077441+0000","last_undegraded":"2024-05-03T17:53:20.077441+0000","last_fullsized":"2024-05-03T17:53:20.077441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:21:17.803027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076565+0000","last_change":"2024-05-03T17:52:35.730547+0000","last_active":"2024-05-03T17:53:20.076565+0000","last_peered":"2024-05-03T17:53:20.076565+0000","last_clean":"2024-05-03T17:53:20.076565+0000","last_became_active":"2024-05-03T17:52:35.729582+0000","last_became_peered":"2024-05-03T17:52:35.729582+0000","last_unstale":"2024-05-03T17:53:20.076565+0000","last_undegraded":"2024-05-03T17:53:20.076565+0000","last_fullsized":"2024-05-03T17:53:20.076565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:24:55.110537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.868010+0000","last_change":"2024-05-03T17:52:35.726305+0000","last_active":"2024-05-03T17:53:19.868010+0000","last_peered":"2024-05-03T17:53:19.868010+0000","last_clean":"2024-05-03T17:53:19.868010+0000","last_became_active":"2024-05-03T17:52:35.725858+0000","last_became_peered":"2024-05-03T17:52:35.725858+0000","last_unstale":"2024-05-03T17:53:19.868010+0000","last_undegraded":"2024-05-03T17:53:19.868010+0000","last_fullsized":"2024-05-03T17:53:19.868010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:59.347307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871846+0000","last_change":"2024-05-03T17:52:35.726865+0000","last_active":"2024-05-03T17:53:19.871846+0000","last_peered":"2024-05-03T17:53:19.871846+0000","last_clean":"2024-05-03T17:53:19.871846+0000","last_became_active":"2024-05-03T17:52:35.726483+0000","last_became_peered":"2024-05-03T17:52:35.726483+0000","last_unstale":"2024-05-03T17:53:19.871846+0000","last_undegraded":"2024-05-03T17:53:19.871846+0000","last_fullsized":"2024-05-03T17:53:19.871846+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:17.851133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077159+0000","last_change":"2024-05-03T17:52:35.730551+0000","last_active":"2024-05-03T17:53:20.077159+0000","last_peered":"2024-05-03T17:53:20.077159+0000","last_clean":"2024-05-03T17:53:20.077159+0000","last_became_active":"2024-05-03T17:52:35.730187+0000","last_became_peered":"2024-05-03T17:52:35.730187+0000","last_unstale":"2024-05-03T17:53:20.077159+0000","last_undegraded":"2024-05-03T17:53:20.077159+0000","last_fullsized":"2024-05-03T17:53:20.077159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:18:56.211061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077071+0000","last_change":"2024-05-03T17:52:35.730040+0000","last_active":"2024-05-03T17:53:20.077071+0000","last_peered":"2024-05-03T17:53:20.077071+0000","last_clean":"2024-05-03T17:53:20.077071+0000","last_became_active":"2024-05-03T17:52:35.729549+0000","last_became_peered":"2024-05-03T17:52:35.729549+0000","last_unstale":"2024-05-03T17:53:20.077071+0000","last_undegraded":"2024-05-03T17:53:20.077071+0000","last_fullsized":"2024-05-03T17:53:20.077071+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:50:05.052247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867289+0000","last_change":"2024-05-03T17:52:35.726709+0000","last_active":"2024-05-03T17:53:19.867289+0000","last_peered":"2024-05-03T17:53:19.867289+0000","last_clean":"2024-05-03T17:53:19.867289+0000","last_became_active":"2024-05-03T17:52:35.726166+0000","last_became_peered":"2024-05-03T17:52:35.726166+0000","last_unstale":"2024-05-03T17:53:19.867289+0000","last_undegraded":"2024-05-03T17:53:19.867289+0000","last_fullsized":"2024-05-03T17:53:19.867289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.164622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870439+0000","last_change":"2024-05-03T17:52:35.727583+0000","last_active":"2024-05-03T17:53:19.870439+0000","last_peered":"2024-05-03T17:53:19.870439+0000","last_clean":"2024-05-03T17:53:19.870439+0000","last_became_active":"2024-05-03T17:52:35.727131+0000","last_became_peered":"2024-05-03T17:52:35.727131+0000","last_unstale":"2024-05-03T17:53:19.870439+0000","last_undegraded":"2024-05-03T17:53:19.870439+0000","last_fullsized":"2024-05-03T17:53:19.870439+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:28:21.255954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.887563+0000","last_change":"2024-05-03T17:53:19.887563+0000","last_active":"2024-05-03T17:53:19.887563+0000","last_peered":"2024-05-03T17:53:19.887563+0000","last_clean":"2024-05-03T17:53:19.887563+0000","last_became_active":"2024-05-03T17:53:19.887068+0000","last_became_peered":"2024-05-03T17:53:19.887068+0000","last_unstale":"2024-05-03T17:53:19.887563+0000","last_undegraded":"2024-05-03T17:53:19.887563+0000","last_fullsized":"2024-05-03T17:53:19.887563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:49:49.354138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867360+0000","last_change":"2024-05-03T17:52:35.717354+0000","last_active":"2024-05-03T17:53:19.867360+0000","last_peered":"2024-05-03T17:53:19.867360+0000","last_clean":"2024-05-03T17:53:19.867360+0000","last_became_active":"2024-05-03T17:52:35.717025+0000","last_became_peered":"2024-05-03T17:52:35.717025+0000","last_unstale":"2024-05-03T17:53:19.867360+0000","last_undegraded":"2024-05-03T17:53:19.867360+0000","last_fullsized":"2024-05-03T17:53:19.867360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:11:07.778484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870387+0000","last_change":"2024-05-03T17:52:35.725626+0000","last_active":"2024-05-03T17:53:19.870387+0000","last_peered":"2024-05-03T17:53:19.870387+0000","last_clean":"2024-05-03T17:53:19.870387+0000","last_became_active":"2024-05-03T17:52:35.725239+0000","last_became_peered":"2024-05-03T17:52:35.725239+0000","last_unstale":"2024-05-03T17:53:19.870387+0000","last_undegraded":"2024-05-03T17:53:19.870387+0000","last_fullsized":"2024-05-03T17:53:19.870387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:44:18.359484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869622+0000","last_change":"2024-05-03T17:52:35.724250+0000","last_active":"2024-05-03T17:53:19.869622+0000","last_peered":"2024-05-03T17:53:19.869622+0000","last_clean":"2024-05-03T17:53:19.869622+0000","last_became_active":"2024-05-03T17:52:35.723998+0000","last_became_peered":"2024-05-03T17:52:35.723998+0000","last_unstale":"2024-05-03T17:53:19.869622+0000","last_undegraded":"2024-05-03T17:53:19.869622+0000","last_fullsized":"2024-05-03T17:53:19.869622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:03:32.939559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886262+0000","last_change":"2024-05-03T17:52:35.726237+0000","last_active":"2024-05-03T17:53:19.886262+0000","last_peered":"2024-05-03T17:53:19.886262+0000","last_clean":"2024-05-03T17:53:19.886262+0000","last_became_active":"2024-05-03T17:52:35.725847+0000","last_became_peered":"2024-05-03T17:52:35.725847+0000","last_unstale":"2024-05-03T17:53:19.886262+0000","last_undegraded":"2024-05-03T17:53:19.886262+0000","last_fullsized":"2024-05-03T17:53:19.886262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:06.472532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869773+0000","last_change":"2024-05-03T17:52:32.687198+0000","last_active":"2024-05-03T17:53:19.869773+0000","last_peered":"2024-05-03T17:53:19.869773+0000","last_clean":"2024-05-03T17:53:19.869773+0000","last_became_active":"2024-05-03T17:52:32.686848+0000","last_became_peered":"2024-05-03T17:52:32.686848+0000","last_unstale":"2024-05-03T17:53:19.869773+0000","last_undegraded":"2024-05-03T17:53:19.869773+0000","last_fullsized":"2024-05-03T17:53:19.869773+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:08:24.244371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076791+0000","last_change":"2024-05-03T17:52:35.730595+0000","last_active":"2024-05-03T17:53:20.076791+0000","last_peered":"2024-05-03T17:53:20.076791+0000","last_clean":"2024-05-03T17:53:20.076791+0000","last_became_active":"2024-05-03T17:52:35.729999+0000","last_became_peered":"2024-05-03T17:52:35.729999+0000","last_unstale":"2024-05-03T17:53:20.076791+0000","last_undegraded":"2024-05-03T17:53:20.076791+0000","last_fullsized":"2024-05-03T17:53:20.076791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:14.485756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076952+0000","last_change":"2024-05-03T17:52:32.935239+0000","last_active":"2024-05-03T17:53:20.076952+0000","last_peered":"2024-05-03T17:53:20.076952+0000","last_clean":"2024-05-03T17:53:20.076952+0000","last_became_active":"2024-05-03T17:52:32.934915+0000","last_became_peered":"2024-05-03T17:52:32.934915+0000","last_unstale":"2024-05-03T17:53:20.076952+0000","last_undegraded":"2024-05-03T17:53:20.076952+0000","last_fullsized":"2024-05-03T17:53:20.076952+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:20.785071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870971+0000","last_change":"2024-05-03T17:52:35.724940+0000","last_active":"2024-05-03T17:53:19.870971+0000","last_peered":"2024-05-03T17:53:19.870971+0000","last_clean":"2024-05-03T17:53:19.870971+0000","last_became_active":"2024-05-03T17:52:35.724603+0000","last_became_peered":"2024-05-03T17:52:35.724603+0000","last_unstale":"2024-05-03T17:53:19.870971+0000","last_undegraded":"2024-05-03T17:53:19.870971+0000","last_fullsized":"2024-05-03T17:53:19.870971+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:43.640499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871451+0000","last_change":"2024-05-03T17:52:32.688202+0000","last_active":"2024-05-03T17:53:19.871451+0000","last_peered":"2024-05-03T17:53:19.871451+0000","last_clean":"2024-05-03T17:53:19.871451+0000","last_became_active":"2024-05-03T17:52:32.687893+0000","last_became_peered":"2024-05-03T17:52:32.687893+0000","last_unstale":"2024-05-03T17:53:19.871451+0000","last_undegraded":"2024-05-03T17:53:19.871451+0000","last_fullsized":"2024-05-03T17:53:19.871451+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:43.342832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867555+0000","last_change":"2024-05-03T17:52:35.720980+0000","last_active":"2024-05-03T17:53:19.867555+0000","last_peered":"2024-05-03T17:53:19.867555+0000","last_clean":"2024-05-03T17:53:19.867555+0000","last_became_active":"2024-05-03T17:52:35.720582+0000","last_became_peered":"2024-05-03T17:52:35.720582+0000","last_unstale":"2024-05-03T17:53:19.867555+0000","last_undegraded":"2024-05-03T17:53:19.867555+0000","last_fullsized":"2024-05-03T17:53:19.867555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:01:22.938186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.877543+0000","last_change":"2024-05-03T17:53:19.877543+0000","last_active":"2024-05-03T17:53:19.877543+0000","last_peered":"2024-05-03T17:53:19.877543+0000","last_clean":"2024-05-03T17:53:19.877543+0000","last_became_active":"2024-05-03T17:53:19.873988+0000","last_became_peered":"2024-05-03T17:53:19.873988+0000","last_unstale":"2024-05-03T17:53:19.877543+0000","last_undegraded":"2024-05-03T17:53:19.877543+0000","last_fullsized":"2024-05-03T17:53:19.877543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:49:49.105623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.079283+0000","last_change":"2024-05-03T17:53:20.078906+0000","last_active":"2024-05-03T17:53:20.079283+0000","last_peered":"2024-05-03T17:53:20.079283+0000","last_clean":"2024-05-03T17:53:20.079283+0000","last_became_active":"2024-05-03T17:53:20.078538+0000","last_became_peered":"2024-05-03T17:53:20.078538+0000","last_unstale":"2024-05-03T17:53:20.079283+0000","last_undegraded":"2024-05-03T17:53:20.079283+0000","last_fullsized":"2024-05-03T17:53:20.079283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:59:46.945417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":145,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:42.916580+0000","last_change":"2024-05-03T17:52:32.692735+0000","last_active":"2024-05-03T17:54:42.916580+0000","last_peered":"2024-05-03T17:54:42.916580+0000","last_clean":"2024-05-03T17:54:42.916580+0000","last_became_active":"2024-05-03T17:52:32.692394+0000","last_became_peered":"2024-05-03T17:52:32.692394+0000","last_unstale":"2024-05-03T17:54:42.916580+0000","last_undegraded":"2024-05-03T17:54:42.916580+0000","last_fullsized":"2024-05-03T17:54:42.916580+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:40:11.434751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870798+0000","last_change":"2024-05-03T17:52:35.727167+0000","last_active":"2024-05-03T17:53:19.870798+0000","last_peered":"2024-05-03T17:53:19.870798+0000","last_clean":"2024-05-03T17:53:19.870798+0000","last_became_active":"2024-05-03T17:52:35.726792+0000","last_became_peered":"2024-05-03T17:52:35.726792+0000","last_unstale":"2024-05-03T17:53:19.870798+0000","last_undegraded":"2024-05-03T17:53:19.870798+0000","last_fullsized":"2024-05-03T17:53:19.870798+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:54:01.823025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871452+0000","last_change":"2024-05-03T17:52:32.687573+0000","last_active":"2024-05-03T17:53:19.871452+0000","last_peered":"2024-05-03T17:53:19.871452+0000","last_clean":"2024-05-03T17:53:19.871452+0000","last_became_active":"2024-05-03T17:52:32.686158+0000","last_became_peered":"2024-05-03T17:52:32.686158+0000","last_unstale":"2024-05-03T17:53:19.871452+0000","last_undegraded":"2024-05-03T17:53:19.871452+0000","last_fullsized":"2024-05-03T17:53:19.871452+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:19.891479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873172+0000","last_change":"2024-05-03T17:52:35.728049+0000","last_active":"2024-05-03T17:53:19.873172+0000","last_peered":"2024-05-03T17:53:19.873172+0000","last_clean":"2024-05-03T17:53:19.873172+0000","last_became_active":"2024-05-03T17:52:35.727713+0000","last_became_peered":"2024-05-03T17:52:35.727713+0000","last_unstale":"2024-05-03T17:53:19.873172+0000","last_undegraded":"2024-05-03T17:53:19.873172+0000","last_fullsized":"2024-05-03T17:53:19.873172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:28:01.798457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867118+0000","last_change":"2024-05-03T17:52:32.685679+0000","last_active":"2024-05-03T17:53:19.867118+0000","last_peered":"2024-05-03T17:53:19.867118+0000","last_clean":"2024-05-03T17:53:19.867118+0000","last_became_active":"2024-05-03T17:52:32.685306+0000","last_became_peered":"2024-05-03T17:52:32.685306+0000","last_unstale":"2024-05-03T17:53:19.867118+0000","last_undegraded":"2024-05-03T17:53:19.867118+0000","last_fullsized":"2024-05-03T17:53:19.867118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:48:22.218310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076501+0000","last_change":"2024-05-03T17:52:35.717111+0000","last_active":"2024-05-03T17:53:20.076501+0000","last_peered":"2024-05-03T17:53:20.076501+0000","last_clean":"2024-05-03T17:53:20.076501+0000","last_became_active":"2024-05-03T17:52:35.716782+0000","last_became_peered":"2024-05-03T17:52:35.716782+0000","last_unstale":"2024-05-03T17:53:20.076501+0000","last_undegraded":"2024-05-03T17:53:20.076501+0000","last_fullsized":"2024-05-03T17:53:20.076501+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:31:57.766927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872477+0000","last_change":"2024-05-03T17:52:32.690846+0000","last_active":"2024-05-03T17:53:19.872477+0000","last_peered":"2024-05-03T17:53:19.872477+0000","last_clean":"2024-05-03T17:53:19.872477+0000","last_became_active":"2024-05-03T17:52:32.690506+0000","last_became_peered":"2024-05-03T17:52:32.690506+0000","last_unstale":"2024-05-03T17:53:19.872477+0000","last_undegraded":"2024-05-03T17:53:19.872477+0000","last_fullsized":"2024-05-03T17:53:19.872477+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:19.810205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869511+0000","last_change":"2024-05-03T17:52:35.718229+0000","last_active":"2024-05-03T17:53:19.869511+0000","last_peered":"2024-05-03T17:53:19.869511+0000","last_clean":"2024-05-03T17:53:19.869511+0000","last_became_active":"2024-05-03T17:52:35.717907+0000","last_became_peered":"2024-05-03T17:52:35.717907+0000","last_unstale":"2024-05-03T17:53:19.869511+0000","last_undegraded":"2024-05-03T17:53:19.869511+0000","last_fullsized":"2024-05-03T17:53:19.869511+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:45.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":138,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:40.913073+0000","last_change":"2024-05-03T17:52:32.691174+0000","last_active":"2024-05-03T17:54:40.913073+0000","last_peered":"2024-05-03T17:54:40.913073+0000","last_clean":"2024-05-03T17:54:40.913073+0000","last_became_active":"2024-05-03T17:52:32.689907+0000","last_became_peered":"2024-05-03T17:52:32.689907+0000","last_unstale":"2024-05-03T17:54:40.913073+0000","last_undegraded":"2024-05-03T17:54:40.913073+0000","last_fullsized":"2024-05-03T17:54:40.913073+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:46.630436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867946+0000","last_change":"2024-05-03T17:52:35.720290+0000","last_active":"2024-05-03T17:53:19.867946+0000","last_peered":"2024-05-03T17:53:19.867946+0000","last_clean":"2024-05-03T17:53:19.867946+0000","last_became_active":"2024-05-03T17:52:35.719962+0000","last_became_peered":"2024-05-03T17:52:35.719962+0000","last_unstale":"2024-05-03T17:53:19.867946+0000","last_undegraded":"2024-05-03T17:53:19.867946+0000","last_fullsized":"2024-05-03T17:53:19.867946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:31:41.301592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077710+0000","last_change":"2024-05-03T17:52:32.689937+0000","last_active":"2024-05-03T17:53:20.077710+0000","last_peered":"2024-05-03T17:53:20.077710+0000","last_clean":"2024-05-03T17:53:20.077710+0000","last_became_active":"2024-05-03T17:52:32.689624+0000","last_became_peered":"2024-05-03T17:52:32.689624+0000","last_unstale":"2024-05-03T17:53:20.077710+0000","last_undegraded":"2024-05-03T17:53:20.077710+0000","last_fullsized":"2024-05-03T17:53:20.077710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:24.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870881+0000","last_change":"2024-05-03T17:52:35.726824+0000","last_active":"2024-05-03T17:53:19.870881+0000","last_peered":"2024-05-03T17:53:19.870881+0000","last_clean":"2024-05-03T17:53:19.870881+0000","last_became_active":"2024-05-03T17:52:35.726494+0000","last_became_peered":"2024-05-03T17:52:35.726494+0000","last_unstale":"2024-05-03T17:53:19.870881+0000","last_undegraded":"2024-05-03T17:53:19.870881+0000","last_fullsized":"2024-05-03T17:53:19.870881+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:41:21.284852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873051+0000","last_change":"2024-05-03T17:52:32.935471+0000","last_active":"2024-05-03T17:53:19.873051+0000","last_peered":"2024-05-03T17:53:19.873051+0000","last_clean":"2024-05-03T17:53:19.873051+0000","last_became_active":"2024-05-03T17:52:32.935012+0000","last_became_peered":"2024-05-03T17:52:32.935012+0000","last_unstale":"2024-05-03T17:53:19.873051+0000","last_undegraded":"2024-05-03T17:53:19.873051+0000","last_fullsized":"2024-05-03T17:53:19.873051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:35:38.654577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867634+0000","last_change":"2024-05-03T17:52:35.727124+0000","last_active":"2024-05-03T17:53:19.867634+0000","last_peered":"2024-05-03T17:53:19.867634+0000","last_clean":"2024-05-03T17:53:19.867634+0000","last_became_active":"2024-05-03T17:52:35.726804+0000","last_became_peered":"2024-05-03T17:52:35.726804+0000","last_unstale":"2024-05-03T17:53:19.867634+0000","last_undegraded":"2024-05-03T17:53:19.867634+0000","last_fullsized":"2024-05-03T17:53:19.867634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:32.000803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'29","reported_seq":282,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:41.914320+0000","last_change":"2024-05-03T17:52:32.685014+0000","last_active":"2024-05-03T17:54:41.914320+0000","last_peered":"2024-05-03T17:54:41.914320+0000","last_clean":"2024-05-03T17:54:41.914320+0000","last_became_active":"2024-05-03T17:52:32.684660+0000","last_became_peered":"2024-05-03T17:52:32.684660+0000","last_unstale":"2024-05-03T17:54:41.914320+0000","last_undegraded":"2024-05-03T17:54:41.914320+0000","last_fullsized":"2024-05-03T17:54:41.914320+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:25:56.890283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54817,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":227,"num_read_kb":166,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871281+0000","last_change":"2024-05-03T17:52:35.733675+0000","last_active":"2024-05-03T17:53:19.871281+0000","last_peered":"2024-05-03T17:53:19.871281+0000","last_clean":"2024-05-03T17:53:19.871281+0000","last_became_active":"2024-05-03T17:52:35.733234+0000","last_became_peered":"2024-05-03T17:52:35.733234+0000","last_unstale":"2024-05-03T17:53:19.871281+0000","last_undegraded":"2024-05-03T17:53:19.871281+0000","last_fullsized":"2024-05-03T17:53:19.871281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:57.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886182+0000","last_change":"2024-05-03T17:52:32.690462+0000","last_active":"2024-05-03T17:53:19.886182+0000","last_peered":"2024-05-03T17:53:19.886182+0000","last_clean":"2024-05-03T17:53:19.886182+0000","last_became_active":"2024-05-03T17:52:32.690169+0000","last_became_peered":"2024-05-03T17:52:32.690169+0000","last_unstale":"2024-05-03T17:53:19.886182+0000","last_undegraded":"2024-05-03T17:53:19.886182+0000","last_fullsized":"2024-05-03T17:53:19.886182+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:59.324612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867010+0000","last_change":"2024-05-03T17:52:35.720730+0000","last_active":"2024-05-03T17:53:19.867010+0000","last_peered":"2024-05-03T17:53:19.867010+0000","last_clean":"2024-05-03T17:53:19.867010+0000","last_became_active":"2024-05-03T17:52:35.720392+0000","last_became_peered":"2024-05-03T17:52:35.720392+0000","last_unstale":"2024-05-03T17:53:19.867010+0000","last_undegraded":"2024-05-03T17:53:19.867010+0000","last_fullsized":"2024-05-03T17:53:19.867010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:58.528148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872742+0000","last_change":"2024-05-03T17:52:32.691439+0000","last_active":"2024-05-03T17:53:19.872742+0000","last_peered":"2024-05-03T17:53:19.872742+0000","last_clean":"2024-05-03T17:53:19.872742+0000","last_became_active":"2024-05-03T17:52:32.691080+0000","last_became_peered":"2024-05-03T17:52:32.691080+0000","last_unstale":"2024-05-03T17:53:19.872742+0000","last_undegraded":"2024-05-03T17:53:19.872742+0000","last_fullsized":"2024-05-03T17:53:19.872742+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:40:43.120194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077160+0000","last_change":"2024-05-03T17:52:35.729860+0000","last_active":"2024-05-03T17:53:20.077160+0000","last_peered":"2024-05-03T17:53:20.077160+0000","last_clean":"2024-05-03T17:53:20.077160+0000","last_became_active":"2024-05-03T17:52:35.729508+0000","last_became_peered":"2024-05-03T17:52:35.729508+0000","last_unstale":"2024-05-03T17:53:20.077160+0000","last_undegraded":"2024-05-03T17:53:20.077160+0000","last_fullsized":"2024-05-03T17:53:20.077160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:39:20.859215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":144,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:42.916389+0000","last_change":"2024-05-03T17:52:32.689346+0000","last_active":"2024-05-03T17:54:42.916389+0000","last_peered":"2024-05-03T17:54:42.916389+0000","last_clean":"2024-05-03T17:54:42.916389+0000","last_became_active":"2024-05-03T17:52:32.689009+0000","last_became_peered":"2024-05-03T17:52:32.689009+0000","last_unstale":"2024-05-03T17:54:42.916389+0000","last_undegraded":"2024-05-03T17:54:42.916389+0000","last_fullsized":"2024-05-03T17:54:42.916389+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:55:36.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077075+0000","last_change":"2024-05-03T17:52:35.729226+0000","last_active":"2024-05-03T17:53:20.077075+0000","last_peered":"2024-05-03T17:53:20.077075+0000","last_clean":"2024-05-03T17:53:20.077075+0000","last_became_active":"2024-05-03T17:52:35.728846+0000","last_became_peered":"2024-05-03T17:52:35.728846+0000","last_unstale":"2024-05-03T17:53:20.077075+0000","last_undegraded":"2024-05-03T17:53:20.077075+0000","last_fullsized":"2024-05-03T17:53:20.077075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:39:16.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870410+0000","last_change":"2024-05-03T17:52:32.936108+0000","last_active":"2024-05-03T17:53:19.870410+0000","last_peered":"2024-05-03T17:53:19.870410+0000","last_clean":"2024-05-03T17:53:19.870410+0000","last_became_active":"2024-05-03T17:52:32.935782+0000","last_became_peered":"2024-05-03T17:52:32.935782+0000","last_unstale":"2024-05-03T17:53:19.870410+0000","last_undegraded":"2024-05-03T17:53:19.870410+0000","last_fullsized":"2024-05-03T17:53:19.870410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:47.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872695+0000","last_change":"2024-05-03T17:52:35.727163+0000","last_active":"2024-05-03T17:53:19.872695+0000","last_peered":"2024-05-03T17:53:19.872695+0000","last_clean":"2024-05-03T17:53:19.872695+0000","last_became_active":"2024-05-03T17:52:35.726824+0000","last_became_peered":"2024-05-03T17:52:35.726824+0000","last_unstale":"2024-05-03T17:53:19.872695+0000","last_undegraded":"2024-05-03T17:53:19.872695+0000","last_fullsized":"2024-05-03T17:53:19.872695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:24:50.803860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871546+0000","last_change":"2024-05-03T17:52:32.687329+0000","last_active":"2024-05-03T17:53:19.871546+0000","last_peered":"2024-05-03T17:53:19.871546+0000","last_clean":"2024-05-03T17:53:19.871546+0000","last_became_active":"2024-05-03T17:52:32.687025+0000","last_became_peered":"2024-05-03T17:52:32.687025+0000","last_unstale":"2024-05-03T17:53:19.871546+0000","last_undegraded":"2024-05-03T17:53:19.871546+0000","last_fullsized":"2024-05-03T17:53:19.871546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:42:15.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869955+0000","last_change":"2024-05-03T17:52:35.728797+0000","last_active":"2024-05-03T17:53:19.869955+0000","last_peered":"2024-05-03T17:53:19.869955+0000","last_clean":"2024-05-03T17:53:19.869955+0000","last_became_active":"2024-05-03T17:52:35.728429+0000","last_became_peered":"2024-05-03T17:52:35.728429+0000","last_unstale":"2024-05-03T17:53:19.869955+0000","last_undegraded":"2024-05-03T17:53:19.869955+0000","last_fullsized":"2024-05-03T17:53:19.869955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:19.584593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872992+0000","last_change":"2024-05-03T17:53:19.872508+0000","last_active":"2024-05-03T17:53:19.872992+0000","last_peered":"2024-05-03T17:53:19.872992+0000","last_clean":"2024-05-03T17:53:19.872992+0000","last_became_active":"2024-05-03T17:53:19.872175+0000","last_became_peered":"2024-05-03T17:53:19.872175+0000","last_unstale":"2024-05-03T17:53:19.872992+0000","last_undegraded":"2024-05-03T17:53:19.872992+0000","last_fullsized":"2024-05-03T17:53:19.872992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:24:08.246884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871648+0000","last_change":"2024-05-03T17:52:35.719115+0000","last_active":"2024-05-03T17:53:19.871648+0000","last_peered":"2024-05-03T17:53:19.871648+0000","last_clean":"2024-05-03T17:53:19.871648+0000","last_became_active":"2024-05-03T17:52:35.718054+0000","last_became_peered":"2024-05-03T17:52:35.718054+0000","last_unstale":"2024-05-03T17:53:19.871648+0000","last_undegraded":"2024-05-03T17:53:19.871648+0000","last_fullsized":"2024-05-03T17:53:19.871648+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.903766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077630+0000","last_change":"2024-05-03T17:52:32.688993+0000","last_active":"2024-05-03T17:53:20.077630+0000","last_peered":"2024-05-03T17:53:20.077630+0000","last_clean":"2024-05-03T17:53:20.077630+0000","last_became_active":"2024-05-03T17:52:32.688706+0000","last_became_peered":"2024-05-03T17:52:32.688706+0000","last_unstale":"2024-05-03T17:53:20.077630+0000","last_undegraded":"2024-05-03T17:53:20.077630+0000","last_fullsized":"2024-05-03T17:53:20.077630+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:40:14.023338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873314+0000","last_change":"2024-05-03T17:53:19.872957+0000","last_active":"2024-05-03T17:53:19.873314+0000","last_peered":"2024-05-03T17:53:19.873314+0000","last_clean":"2024-05-03T17:53:19.873314+0000","last_became_active":"2024-05-03T17:53:19.872690+0000","last_became_peered":"2024-05-03T17:53:19.872690+0000","last_unstale":"2024-05-03T17:53:19.873314+0000","last_undegraded":"2024-05-03T17:53:19.873314+0000","last_fullsized":"2024-05-03T17:53:19.873314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:53:43.801983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873043+0000","last_change":"2024-05-03T17:52:35.728958+0000","last_active":"2024-05-03T17:53:19.873043+0000","last_peered":"2024-05-03T17:53:19.873043+0000","last_clean":"2024-05-03T17:53:19.873043+0000","last_became_active":"2024-05-03T17:52:35.728619+0000","last_became_peered":"2024-05-03T17:52:35.728619+0000","last_unstale":"2024-05-03T17:53:19.873043+0000","last_undegraded":"2024-05-03T17:53:19.873043+0000","last_fullsized":"2024-05-03T17:53:19.873043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:29:29.922324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":141,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:41.914507+0000","last_change":"2024-05-03T17:52:32.935367+0000","last_active":"2024-05-03T17:54:41.914507+0000","last_peered":"2024-05-03T17:54:41.914507+0000","last_clean":"2024-05-03T17:54:41.914507+0000","last_became_active":"2024-05-03T17:52:32.934938+0000","last_became_peered":"2024-05-03T17:52:32.934938+0000","last_unstale":"2024-05-03T17:54:41.914507+0000","last_undegraded":"2024-05-03T17:54:41.914507+0000","last_fullsized":"2024-05-03T17:54:41.914507+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:07:21.642883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076712+0000","last_change":"2024-05-03T17:52:35.731035+0000","last_active":"2024-05-03T17:53:20.076712+0000","last_peered":"2024-05-03T17:53:20.076712+0000","last_clean":"2024-05-03T17:53:20.076712+0000","last_became_active":"2024-05-03T17:52:35.730034+0000","last_became_peered":"2024-05-03T17:52:35.730034+0000","last_unstale":"2024-05-03T17:53:20.076712+0000","last_undegraded":"2024-05-03T17:53:20.076712+0000","last_fullsized":"2024-05-03T17:53:20.076712+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:25.545290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077931+0000","last_change":"2024-05-03T17:52:32.690043+0000","last_active":"2024-05-03T17:53:20.077931+0000","last_peered":"2024-05-03T17:53:20.077931+0000","last_clean":"2024-05-03T17:53:20.077931+0000","last_became_active":"2024-05-03T17:52:32.689562+0000","last_became_peered":"2024-05-03T17:52:32.689562+0000","last_unstale":"2024-05-03T17:53:20.077931+0000","last_undegraded":"2024-05-03T17:53:20.077931+0000","last_fullsized":"2024-05-03T17:53:20.077931+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:59.014968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077348+0000","last_change":"2024-05-03T17:51:02.340854+0000","last_active":"2024-05-03T17:53:20.077348+0000","last_peered":"2024-05-03T17:53:20.077348+0000","last_clean":"2024-05-03T17:53:20.077348+0000","last_became_active":"2024-05-03T17:51:02.336454+0000","last_became_peered":"2024-05-03T17:51:02.336454+0000","last_unstale":"2024-05-03T17:53:20.077348+0000","last_undegraded":"2024-05-03T17:53:20.077348+0000","last_fullsized":"2024-05-03T17:53:20.077348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_clean_scrub_stamp":"2024-05-03T17:47:41.306771+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:18:08.512098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.887660+0000","last_change":"2024-05-03T17:53:19.887126+0000","last_active":"2024-05-03T17:53:19.887660+0000","last_peered":"2024-05-03T17:53:19.887660+0000","last_clean":"2024-05-03T17:53:19.887660+0000","last_became_active":"2024-05-03T17:53:19.886736+0000","last_became_peered":"2024-05-03T17:53:19.886736+0000","last_unstale":"2024-05-03T17:53:19.887660+0000","last_undegraded":"2024-05-03T17:53:19.887660+0000","last_fullsized":"2024-05-03T17:53:19.887660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:35:02.036825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077451+0000","last_change":"2024-05-03T17:52:32.690093+0000","last_active":"2024-05-03T17:53:20.077451+0000","last_peered":"2024-05-03T17:53:20.077451+0000","last_clean":"2024-05-03T17:53:20.077451+0000","last_became_active":"2024-05-03T17:52:32.689777+0000","last_became_peered":"2024-05-03T17:52:32.689777+0000","last_unstale":"2024-05-03T17:53:20.077451+0000","last_undegraded":"2024-05-03T17:53:20.077451+0000","last_fullsized":"2024-05-03T17:53:20.077451+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:09.693011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871391+0000","last_change":"2024-05-03T17:52:32.935589+0000","last_active":"2024-05-03T17:53:19.871391+0000","last_peered":"2024-05-03T17:53:19.871391+0000","last_clean":"2024-05-03T17:53:19.871391+0000","last_became_active":"2024-05-03T17:52:32.935221+0000","last_became_peered":"2024-05-03T17:52:32.935221+0000","last_unstale":"2024-05-03T17:53:19.871391+0000","last_undegraded":"2024-05-03T17:53:19.871391+0000","last_fullsized":"2024-05-03T17:53:19.871391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:26:08.001761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867238+0000","last_change":"2024-05-03T17:52:35.724293+0000","last_active":"2024-05-03T17:53:19.867238+0000","last_peered":"2024-05-03T17:53:19.867238+0000","last_clean":"2024-05-03T17:53:19.867238+0000","last_became_active":"2024-05-03T17:52:35.723978+0000","last_became_peered":"2024-05-03T17:52:35.723978+0000","last_unstale":"2024-05-03T17:53:19.867238+0000","last_undegraded":"2024-05-03T17:53:19.867238+0000","last_fullsized":"2024-05-03T17:53:19.867238+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:23.357872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'10","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:19.733474+0000","last_change":"2024-05-03T17:52:32.687303+0000","last_active":"2024-05-03T17:54:19.733474+0000","last_peered":"2024-05-03T17:54:19.733474+0000","last_clean":"2024-05-03T17:54:19.733474+0000","last_became_active":"2024-05-03T17:52:32.686827+0000","last_became_peered":"2024-05-03T17:52:32.686827+0000","last_unstale":"2024-05-03T17:54:19.733474+0000","last_undegraded":"2024-05-03T17:54:19.733474+0000","last_fullsized":"2024-05-03T17:54:19.733474+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:11:53.565215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076856+0000","last_change":"2024-05-03T17:52:35.720898+0000","last_active":"2024-05-03T17:53:20.076856+0000","last_peered":"2024-05-03T17:53:20.076856+0000","last_clean":"2024-05-03T17:53:20.076856+0000","last_became_active":"2024-05-03T17:52:35.720572+0000","last_became_peered":"2024-05-03T17:52:35.720572+0000","last_unstale":"2024-05-03T17:53:20.076856+0000","last_undegraded":"2024-05-03T17:53:20.076856+0000","last_fullsized":"2024-05-03T17:53:20.076856+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:01.697167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077417+0000","last_change":"2024-05-03T17:52:33.249970+0000","last_active":"2024-05-03T17:53:20.077417+0000","last_peered":"2024-05-03T17:53:20.077417+0000","last_clean":"2024-05-03T17:53:20.077417+0000","last_became_active":"2024-05-03T17:52:33.249596+0000","last_became_peered":"2024-05-03T17:52:33.249596+0000","last_unstale":"2024-05-03T17:53:20.077417+0000","last_undegraded":"2024-05-03T17:53:20.077417+0000","last_fullsized":"2024-05-03T17:53:20.077417+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:46:53.058586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870807+0000","last_change":"2024-05-03T17:52:35.722129+0000","last_active":"2024-05-03T17:53:19.870807+0000","last_peered":"2024-05-03T17:53:19.870807+0000","last_clean":"2024-05-03T17:53:19.870807+0000","last_became_active":"2024-05-03T17:52:35.721815+0000","last_became_peered":"2024-05-03T17:52:35.721815+0000","last_unstale":"2024-05-03T17:53:19.870807+0000","last_undegraded":"2024-05-03T17:53:19.870807+0000","last_fullsized":"2024-05-03T17:53:19.870807+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:33.781350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867452+0000","last_change":"2024-05-03T17:52:32.936157+0000","last_active":"2024-05-03T17:53:19.867452+0000","last_peered":"2024-05-03T17:53:19.867452+0000","last_clean":"2024-05-03T17:53:19.867452+0000","last_became_active":"2024-05-03T17:52:32.935842+0000","last_became_peered":"2024-05-03T17:52:32.935842+0000","last_unstale":"2024-05-03T17:53:19.867452+0000","last_undegraded":"2024-05-03T17:53:19.867452+0000","last_fullsized":"2024-05-03T17:53:19.867452+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:18.006451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870494+0000","last_change":"2024-05-03T17:52:35.735993+0000","last_active":"2024-05-03T17:53:19.870494+0000","last_peered":"2024-05-03T17:53:19.870494+0000","last_clean":"2024-05-03T17:53:19.870494+0000","last_became_active":"2024-05-03T17:52:35.735612+0000","last_became_peered":"2024-05-03T17:52:35.735612+0000","last_unstale":"2024-05-03T17:53:19.870494+0000","last_undegraded":"2024-05-03T17:53:19.870494+0000","last_fullsized":"2024-05-03T17:53:19.870494+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:59.216022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076687+0000","last_change":"2024-05-03T17:52:32.687147+0000","last_active":"2024-05-03T17:53:20.076687+0000","last_peered":"2024-05-03T17:53:20.076687+0000","last_clean":"2024-05-03T17:53:20.076687+0000","last_became_active":"2024-05-03T17:52:32.686831+0000","last_became_peered":"2024-05-03T17:52:32.686831+0000","last_unstale":"2024-05-03T17:53:20.076687+0000","last_undegraded":"2024-05-03T17:53:20.076687+0000","last_fullsized":"2024-05-03T17:53:20.076687+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:24:30.092829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869678+0000","last_change":"2024-05-03T17:52:35.721862+0000","last_active":"2024-05-03T17:53:19.869678+0000","last_peered":"2024-05-03T17:53:19.869678+0000","last_clean":"2024-05-03T17:53:19.869678+0000","last_became_active":"2024-05-03T17:52:35.721393+0000","last_became_peered":"2024-05-03T17:52:35.721393+0000","last_unstale":"2024-05-03T17:53:19.869678+0000","last_undegraded":"2024-05-03T17:53:19.869678+0000","last_fullsized":"2024-05-03T17:53:19.869678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:47:32.014817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886007+0000","last_change":"2024-05-03T17:52:32.687867+0000","last_active":"2024-05-03T17:53:19.886007+0000","last_peered":"2024-05-03T17:53:19.886007+0000","last_clean":"2024-05-03T17:53:19.886007+0000","last_became_active":"2024-05-03T17:52:32.687557+0000","last_became_peered":"2024-05-03T17:52:32.687557+0000","last_unstale":"2024-05-03T17:53:19.886007+0000","last_undegraded":"2024-05-03T17:53:19.886007+0000","last_fullsized":"2024-05-03T17:53:19.886007+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:45.863180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076840+0000","last_change":"2024-05-03T17:52:35.731057+0000","last_active":"2024-05-03T17:53:20.076840+0000","last_peered":"2024-05-03T17:53:20.076840+0000","last_clean":"2024-05-03T17:53:20.076840+0000","last_became_active":"2024-05-03T17:52:35.729293+0000","last_became_peered":"2024-05-03T17:52:35.729293+0000","last_unstale":"2024-05-03T17:53:20.076840+0000","last_undegraded":"2024-05-03T17:53:20.076840+0000","last_fullsized":"2024-05-03T17:53:20.076840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:12:53.888078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870721+0000","last_change":"2024-05-03T17:52:32.689675+0000","last_active":"2024-05-03T17:53:19.870721+0000","last_peered":"2024-05-03T17:53:19.870721+0000","last_clean":"2024-05-03T17:53:19.870721+0000","last_became_active":"2024-05-03T17:52:32.689340+0000","last_became_peered":"2024-05-03T17:52:32.689340+0000","last_unstale":"2024-05-03T17:53:19.870721+0000","last_undegraded":"2024-05-03T17:53:19.870721+0000","last_fullsized":"2024-05-03T17:53:19.870721+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:31.398045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077600+0000","last_change":"2024-05-03T17:52:35.728228+0000","last_active":"2024-05-03T17:53:20.077600+0000","last_peered":"2024-05-03T17:53:20.077600+0000","last_clean":"2024-05-03T17:53:20.077600+0000","last_became_active":"2024-05-03T17:52:35.727233+0000","last_became_peered":"2024-05-03T17:52:35.727233+0000","last_unstale":"2024-05-03T17:53:20.077600+0000","last_undegraded":"2024-05-03T17:53:20.077600+0000","last_fullsized":"2024-05-03T17:53:20.077600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:57.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886416+0000","last_change":"2024-05-03T17:52:33.249690+0000","last_active":"2024-05-03T17:53:19.886416+0000","last_peered":"2024-05-03T17:53:19.886416+0000","last_clean":"2024-05-03T17:53:19.886416+0000","last_became_active":"2024-05-03T17:52:33.249302+0000","last_became_peered":"2024-05-03T17:52:33.249302+0000","last_unstale":"2024-05-03T17:53:19.886416+0000","last_undegraded":"2024-05-03T17:53:19.886416+0000","last_fullsized":"2024-05-03T17:53:19.886416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:05:48.700670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077737+0000","last_change":"2024-05-03T17:52:35.727171+0000","last_active":"2024-05-03T17:53:20.077737+0000","last_peered":"2024-05-03T17:53:20.077737+0000","last_clean":"2024-05-03T17:53:20.077737+0000","last_became_active":"2024-05-03T17:52:35.726834+0000","last_became_peered":"2024-05-03T17:52:35.726834+0000","last_unstale":"2024-05-03T17:53:20.077737+0000","last_undegraded":"2024-05-03T17:53:20.077737+0000","last_fullsized":"2024-05-03T17:53:20.077737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:40:09.938183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872806+0000","last_change":"2024-05-03T17:52:32.693694+0000","last_active":"2024-05-03T17:53:19.872806+0000","last_peered":"2024-05-03T17:53:19.872806+0000","last_clean":"2024-05-03T17:53:19.872806+0000","last_became_active":"2024-05-03T17:52:32.693310+0000","last_became_peered":"2024-05-03T17:52:32.693310+0000","last_unstale":"2024-05-03T17:53:19.872806+0000","last_undegraded":"2024-05-03T17:53:19.872806+0000","last_fullsized":"2024-05-03T17:53:19.872806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:59:37.449434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077538+0000","last_change":"2024-05-03T17:52:35.728267+0000","last_active":"2024-05-03T17:53:20.077538+0000","last_peered":"2024-05-03T17:53:20.077538+0000","last_clean":"2024-05-03T17:53:20.077538+0000","last_became_active":"2024-05-03T17:52:35.727902+0000","last_became_peered":"2024-05-03T17:52:35.727902+0000","last_unstale":"2024-05-03T17:53:20.077538+0000","last_undegraded":"2024-05-03T17:53:20.077538+0000","last_fullsized":"2024-05-03T17:53:20.077538+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:09.223850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867146+0000","last_change":"2024-05-03T17:52:32.687248+0000","last_active":"2024-05-03T17:53:19.867146+0000","last_peered":"2024-05-03T17:53:19.867146+0000","last_clean":"2024-05-03T17:53:19.867146+0000","last_became_active":"2024-05-03T17:52:32.686921+0000","last_became_peered":"2024-05-03T17:52:32.686921+0000","last_unstale":"2024-05-03T17:53:19.867146+0000","last_undegraded":"2024-05-03T17:53:19.867146+0000","last_fullsized":"2024-05-03T17:53:19.867146+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:35:40.849484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077858+0000","last_change":"2024-05-03T17:52:35.729383+0000","last_active":"2024-05-03T17:53:20.077858+0000","last_peered":"2024-05-03T17:53:20.077858+0000","last_clean":"2024-05-03T17:53:20.077858+0000","last_became_active":"2024-05-03T17:52:35.729037+0000","last_became_peered":"2024-05-03T17:52:35.729037+0000","last_unstale":"2024-05-03T17:53:20.077858+0000","last_undegraded":"2024-05-03T17:53:20.077858+0000","last_fullsized":"2024-05-03T17:53:20.077858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:03:03.815770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869889+0000","last_change":"2024-05-03T17:52:35.731809+0000","last_active":"2024-05-03T17:53:19.869889+0000","last_peered":"2024-05-03T17:53:19.869889+0000","last_clean":"2024-05-03T17:53:19.869889+0000","last_became_active":"2024-05-03T17:52:35.729360+0000","last_became_peered":"2024-05-03T17:52:35.729360+0000","last_unstale":"2024-05-03T17:53:19.869889+0000","last_undegraded":"2024-05-03T17:53:19.869889+0000","last_fullsized":"2024-05-03T17:53:19.869889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:22.181184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886214+0000","last_change":"2024-05-03T17:52:35.727771+0000","last_active":"2024-05-03T17:53:19.886214+0000","last_peered":"2024-05-03T17:53:19.886214+0000","last_clean":"2024-05-03T17:53:19.886214+0000","last_became_active":"2024-05-03T17:52:35.727414+0000","last_became_peered":"2024-05-03T17:52:35.727414+0000","last_unstale":"2024-05-03T17:53:19.886214+0000","last_undegraded":"2024-05-03T17:53:19.886214+0000","last_fullsized":"2024-05-03T17:53:19.886214+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:09:02.905481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885945+0000","last_change":"2024-05-03T17:52:35.727433+0000","last_active":"2024-05-03T17:53:19.885945+0000","last_peered":"2024-05-03T17:53:19.885945+0000","last_clean":"2024-05-03T17:53:19.885945+0000","last_became_active":"2024-05-03T17:52:35.727123+0000","last_became_peered":"2024-05-03T17:52:35.727123+0000","last_unstale":"2024-05-03T17:53:19.885945+0000","last_undegraded":"2024-05-03T17:53:19.885945+0000","last_fullsized":"2024-05-03T17:53:19.885945+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:41:36.871788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870777+0000","last_change":"2024-05-03T17:52:35.721332+0000","last_active":"2024-05-03T17:53:19.870777+0000","last_peered":"2024-05-03T17:53:19.870777+0000","last_clean":"2024-05-03T17:53:19.870777+0000","last_became_active":"2024-05-03T17:52:35.721004+0000","last_became_peered":"2024-05-03T17:52:35.721004+0000","last_unstale":"2024-05-03T17:53:19.870777+0000","last_undegraded":"2024-05-03T17:53:19.870777+0000","last_fullsized":"2024-05-03T17:53:19.870777+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:35:03.443163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867305+0000","last_change":"2024-05-03T17:52:35.723649+0000","last_active":"2024-05-03T17:53:19.867305+0000","last_peered":"2024-05-03T17:53:19.867305+0000","last_clean":"2024-05-03T17:53:19.867305+0000","last_became_active":"2024-05-03T17:52:35.723303+0000","last_became_peered":"2024-05-03T17:52:35.723303+0000","last_unstale":"2024-05-03T17:53:19.867305+0000","last_undegraded":"2024-05-03T17:53:19.867305+0000","last_fullsized":"2024-05-03T17:53:19.867305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:02:49.093039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870096+0000","last_change":"2024-05-03T17:52:35.732505+0000","last_active":"2024-05-03T17:53:19.870096+0000","last_peered":"2024-05-03T17:53:19.870096+0000","last_clean":"2024-05-03T17:53:19.870096+0000","last_became_active":"2024-05-03T17:52:35.732180+0000","last_became_peered":"2024-05-03T17:52:35.732180+0000","last_unstale":"2024-05-03T17:53:19.870096+0000","last_undegraded":"2024-05-03T17:53:19.870096+0000","last_fullsized":"2024-05-03T17:53:19.870096+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.371260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886392+0000","last_change":"2024-05-03T17:52:35.732217+0000","last_active":"2024-05-03T17:53:19.886392+0000","last_peered":"2024-05-03T17:53:19.886392+0000","last_clean":"2024-05-03T17:53:19.886392+0000","last_became_active":"2024-05-03T17:52:35.731868+0000","last_became_peered":"2024-05-03T17:52:35.731868+0000","last_unstale":"2024-05-03T17:53:19.886392+0000","last_undegraded":"2024-05-03T17:53:19.886392+0000","last_fullsized":"2024-05-03T17:53:19.886392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:04:17.521528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076813+0000","last_change":"2024-05-03T17:52:35.723641+0000","last_active":"2024-05-03T17:53:20.076813+0000","last_peered":"2024-05-03T17:53:20.076813+0000","last_clean":"2024-05-03T17:53:20.076813+0000","last_became_active":"2024-05-03T17:52:35.723323+0000","last_became_peered":"2024-05-03T17:52:35.723323+0000","last_unstale":"2024-05-03T17:53:20.076813+0000","last_undegraded":"2024-05-03T17:53:20.076813+0000","last_fullsized":"2024-05-03T17:53:20.076813+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:45:34.032407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872477+0000","last_change":"2024-05-03T17:52:35.731008+0000","last_active":"2024-05-03T17:53:19.872477+0000","last_peered":"2024-05-03T17:53:19.872477+0000","last_clean":"2024-05-03T17:53:19.872477+0000","last_became_active":"2024-05-03T17:52:35.730627+0000","last_became_peered":"2024-05-03T17:52:35.730627+0000","last_unstale":"2024-05-03T17:53:19.872477+0000","last_undegraded":"2024-05-03T17:53:19.872477+0000","last_fullsized":"2024-05-03T17:53:19.872477+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:30:32.884518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076519+0000","last_change":"2024-05-03T17:52:35.726938+0000","last_active":"2024-05-03T17:53:20.076519+0000","last_peered":"2024-05-03T17:53:20.076519+0000","last_clean":"2024-05-03T17:53:20.076519+0000","last_became_active":"2024-05-03T17:52:35.726611+0000","last_became_peered":"2024-05-03T17:52:35.726611+0000","last_unstale":"2024-05-03T17:53:20.076519+0000","last_undegraded":"2024-05-03T17:53:20.076519+0000","last_fullsized":"2024-05-03T17:53:20.076519+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:11.587035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867632+0000","last_change":"2024-05-03T17:52:35.722562+0000","last_active":"2024-05-03T17:53:19.867632+0000","last_peered":"2024-05-03T17:53:19.867632+0000","last_clean":"2024-05-03T17:53:19.867632+0000","last_became_active":"2024-05-03T17:52:35.722229+0000","last_became_peered":"2024-05-03T17:52:35.722229+0000","last_unstale":"2024-05-03T17:53:19.867632+0000","last_undegraded":"2024-05-03T17:53:19.867632+0000","last_fullsized":"2024-05-03T17:53:19.867632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:57:06.730533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867188+0000","last_change":"2024-05-03T17:52:35.722707+0000","last_active":"2024-05-03T17:53:19.867188+0000","last_peered":"2024-05-03T17:53:19.867188+0000","last_clean":"2024-05-03T17:53:19.867188+0000","last_became_active":"2024-05-03T17:52:35.721847+0000","last_became_peered":"2024-05-03T17:52:35.721847+0000","last_unstale":"2024-05-03T17:53:19.867188+0000","last_undegraded":"2024-05-03T17:53:19.867188+0000","last_fullsized":"2024-05-03T17:53:19.867188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:56:21.620074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870547+0000","last_change":"2024-05-03T17:52:35.731810+0000","last_active":"2024-05-03T17:53:19.870547+0000","last_peered":"2024-05-03T17:53:19.870547+0000","last_clean":"2024-05-03T17:53:19.870547+0000","last_became_active":"2024-05-03T17:52:35.728986+0000","last_became_peered":"2024-05-03T17:52:35.728986+0000","last_unstale":"2024-05-03T17:53:19.870547+0000","last_undegraded":"2024-05-03T17:53:19.870547+0000","last_fullsized":"2024-05-03T17:53:19.870547+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:18.275792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280811,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59209,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":688,"num_read_kb":405,"num_write":164,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":177627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743467,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":417972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1064661,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070518,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":418208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266746,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":418128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":1010449,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626500,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":417597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":913408,"data_stored":472207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":418868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":473948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":872448,"data_stored":417908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986255,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":860160,"data_stored":418425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822730,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":1009742,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T17:54:49.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T17:54:49.370 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:49.611 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:49.611 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:49.611 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:49 smithi122 ceph-mon[29744]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:54:49.611 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:54:49.611 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:54:49.611 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:49.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:49.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:49.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:49 smithi192 ceph-mon[33753]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:54:49.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:54:49.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:54:49.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:49.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:49.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:49.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:49 smithi149 ceph-mon[33409]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:54:49.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:54:49.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:54:49.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:54:50.443 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:50 smithi122 ceph-mon[29744]: pgmap v354: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:50.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:50 smithi192 ceph-mon[33753]: pgmap v354: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:50.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:50 smithi149 ceph-mon[33409]: pgmap v354: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:54:51.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.011+0000 7f9995aa2700 1 -- 172.21.15.122:0/1520385165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9990104160 msgr2=0x7f9990108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.011+0000 7f9995aa2700 1 --2- 172.21.15.122:0/1520385165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9990104160 0x7f9990108150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f99840099c0 tx=0x7f9984009cd0 comp rx=0 tx=0).stop 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.011+0000 7f9995aa2700 1 -- 172.21.15.122:0/1520385165 shutdown_connections 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.011+0000 7f9995aa2700 1 --2- 172.21.15.122:0/1520385165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9990108880 0x7f9990110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.011+0000 7f9995aa2700 1 --2- 172.21.15.122:0/1520385165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99901037b0 0x7f9990103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.011+0000 7f9995aa2700 1 --2- 172.21.15.122:0/1520385165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9990104160 0x7f9990108150 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.011+0000 7f9995aa2700 1 -- 172.21.15.122:0/1520385165 >> 172.21.15.122:0/1520385165 conn(0x7f999007a580 msgr2=0x7f999007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.011+0000 7f9995aa2700 1 -- 172.21.15.122:0/1520385165 shutdown_connections 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.011+0000 7f9995aa2700 1 -- 172.21.15.122:0/1520385165 wait complete. 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f9995aa2700 1 Processor -- start 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f9995aa2700 1 -- start start 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f9995aa2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99901037b0 0x7f999019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f9995aa2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9990104160 0x7f999019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f9995aa2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9990108880 0x7f9990199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f9995aa2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9990113640 con 0x7f99901037b0 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f9995aa2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f99901134c0 con 0x7f9990104160 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f9995aa2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f99901137c0 con 0x7f9990108880 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f998f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99901037b0 0x7f999019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f998f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99901037b0 0x7f999019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49748/0 (socket says 172.21.15.122:49748) 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f998f7fe700 1 -- 172.21.15.122:0/4137646570 learned_addr learned my addr 172.21.15.122:0/4137646570 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f998effd700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9990104160 0x7f999019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f998ffff700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9990108880 0x7f9990199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f998f7fe700 1 -- 172.21.15.122:0/4137646570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9990108880 msgr2=0x7f9990199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f998f7fe700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9990108880 0x7f9990199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f998f7fe700 1 -- 172.21.15.122:0/4137646570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9990104160 msgr2=0x7f999019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f998f7fe700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9990104160 0x7f999019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.012+0000 7f998f7fe700 1 -- 172.21.15.122:0/4137646570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999019a0e0 con 0x7f99901037b0 2024-05-03T17:54:51.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.013+0000 7f998f7fe700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99901037b0 0x7f999019f7c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f998000d950 tx=0x7f998000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:51.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.013+0000 7f998cff9700 1 -- 172.21.15.122:0/4137646570 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980016070 con 0x7f99901037b0 2024-05-03T17:54:51.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.013+0000 7f9995aa2700 1 -- 172.21.15.122:0/4137646570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999019a3d0 con 0x7f99901037b0 2024-05-03T17:54:51.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.013+0000 7f998cff9700 1 -- 172.21.15.122:0/4137646570 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9980009df0 con 0x7f99901037b0 2024-05-03T17:54:51.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.013+0000 7f998cff9700 1 -- 172.21.15.122:0/4137646570 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998000f3c0 con 0x7f99901037b0 2024-05-03T17:54:51.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.013+0000 7f9995aa2700 1 -- 172.21.15.122:0/4137646570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99901a6690 con 0x7f99901037b0 2024-05-03T17:54:51.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.014+0000 7f9995aa2700 1 -- 172.21.15.122:0/4137646570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999006a190 con 0x7f99901037b0 2024-05-03T17:54:51.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.015+0000 7f998cff9700 1 -- 172.21.15.122:0/4137646570 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f9980019070 con 0x7f99901037b0 2024-05-03T17:54:51.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.016+0000 7f998cff9700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f99780b2910 0x7f99780b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:51.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.016+0000 7f998effd700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f99780b2910 0x7f99780b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:51.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.016+0000 7f998cff9700 1 -- 172.21.15.122:0/4137646570 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f99800e1f90 con 0x7f99901037b0 2024-05-03T17:54:51.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.016+0000 7f998effd700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f99780b2910 0x7f99780b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9984009990 tx=0x7f9984005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.020+0000 7f998cff9700 1 -- 172.21.15.122:0/4137646570 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f99800a9b50 con 0x7f99901037b0 2024-05-03T17:54:51.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.221+0000 7f9995aa2700 1 -- 172.21.15.122:0/4137646570 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99900638e0 con 0x7f99780b2910 2024-05-03T17:54:51.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.228+0000 7f998cff9700 1 -- 172.21.15.122:0/4137646570 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258870 (secure 0 0 0) 0x7f99900638e0 con 0x7f99780b2910 2024-05-03T17:54:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:54:51.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.231+0000 7f9995aa2700 1 -- 172.21.15.122:0/4137646570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f99780b2910 msgr2=0x7f99780b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:51.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.231+0000 7f9995aa2700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f99780b2910 0x7f99780b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9984009990 tx=0x7f9984005c40 comp rx=0 tx=0).stop 2024-05-03T17:54:51.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.231+0000 7f9995aa2700 1 -- 172.21.15.122:0/4137646570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99901037b0 msgr2=0x7f999019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:51.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.231+0000 7f9995aa2700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99901037b0 0x7f999019f7c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f998000d950 tx=0x7f998000dc60 comp rx=0 tx=0).stop 2024-05-03T17:54:51.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.231+0000 7f9995aa2700 1 -- 172.21.15.122:0/4137646570 shutdown_connections 2024-05-03T17:54:51.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.231+0000 7f9995aa2700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f99780b2910 0x7f99780b4dd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:51.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.231+0000 7f9995aa2700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9990108880 0x7f9990199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:51.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.231+0000 7f9995aa2700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99901037b0 0x7f999019f7c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:51.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.231+0000 7f9995aa2700 1 --2- 172.21.15.122:0/4137646570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9990104160 0x7f999019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:51.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.231+0000 7f9995aa2700 1 -- 172.21.15.122:0/4137646570 >> 172.21.15.122:0/4137646570 conn(0x7f999007a580 msgr2=0x7f9990101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:51.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.232+0000 7f9995aa2700 1 -- 172.21.15.122:0/4137646570 shutdown_connections 2024-05-03T17:54:51.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:51.232+0000 7f9995aa2700 1 -- 172.21.15.122:0/4137646570 wait complete. 2024-05-03T17:54:51.231 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-05-03T17:54:51.871 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-05-03T17:54:50.571269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649811,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":794,"num_read_kb":484,"num_write":289,"num_write_kb":2258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376344,"kb_used_data":10500,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565480,"statfs":{"total":1151940427776,"available":1151555051520,"internally_reserved":0,"allocated":10752000,"data_stored":6958113,"data_compressed":107130,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003476"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870216+0000","last_change":"2024-05-03T17:52:35.725057+0000","last_active":"2024-05-03T17:53:19.870216+0000","last_peered":"2024-05-03T17:53:19.870216+0000","last_clean":"2024-05-03T17:53:19.870216+0000","last_became_active":"2024-05-03T17:52:35.724632+0000","last_became_peered":"2024-05-03T17:52:35.724632+0000","last_unstale":"2024-05-03T17:53:19.870216+0000","last_undegraded":"2024-05-03T17:53:19.870216+0000","last_fullsized":"2024-05-03T17:53:19.870216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:46.065357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870713+0000","last_change":"2024-05-03T17:52:35.728500+0000","last_active":"2024-05-03T17:53:19.870713+0000","last_peered":"2024-05-03T17:53:19.870713+0000","last_clean":"2024-05-03T17:53:19.870713+0000","last_became_active":"2024-05-03T17:52:35.728210+0000","last_became_peered":"2024-05-03T17:52:35.728210+0000","last_unstale":"2024-05-03T17:53:19.870713+0000","last_undegraded":"2024-05-03T17:53:19.870713+0000","last_fullsized":"2024-05-03T17:53:19.870713+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:33:23.410617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870687+0000","last_change":"2024-05-03T17:52:35.726300+0000","last_active":"2024-05-03T17:53:19.870687+0000","last_peered":"2024-05-03T17:53:19.870687+0000","last_clean":"2024-05-03T17:53:19.870687+0000","last_became_active":"2024-05-03T17:52:35.725994+0000","last_became_peered":"2024-05-03T17:52:35.725994+0000","last_unstale":"2024-05-03T17:53:19.870687+0000","last_undegraded":"2024-05-03T17:53:19.870687+0000","last_fullsized":"2024-05-03T17:53:19.870687+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:31:19.723453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077329+0000","last_change":"2024-05-03T17:52:35.728105+0000","last_active":"2024-05-03T17:53:20.077329+0000","last_peered":"2024-05-03T17:53:20.077329+0000","last_clean":"2024-05-03T17:53:20.077329+0000","last_became_active":"2024-05-03T17:52:35.727762+0000","last_became_peered":"2024-05-03T17:52:35.727762+0000","last_unstale":"2024-05-03T17:53:20.077329+0000","last_undegraded":"2024-05-03T17:53:20.077329+0000","last_fullsized":"2024-05-03T17:53:20.077329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.640057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870597+0000","last_change":"2024-05-03T17:52:35.734516+0000","last_active":"2024-05-03T17:53:19.870597+0000","last_peered":"2024-05-03T17:53:19.870597+0000","last_clean":"2024-05-03T17:53:19.870597+0000","last_became_active":"2024-05-03T17:52:35.734125+0000","last_became_peered":"2024-05-03T17:52:35.734125+0000","last_unstale":"2024-05-03T17:53:19.870597+0000","last_undegraded":"2024-05-03T17:53:19.870597+0000","last_fullsized":"2024-05-03T17:53:19.870597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:39:25.103137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871539+0000","last_change":"2024-05-03T17:52:35.723427+0000","last_active":"2024-05-03T17:53:19.871539+0000","last_peered":"2024-05-03T17:53:19.871539+0000","last_clean":"2024-05-03T17:53:19.871539+0000","last_became_active":"2024-05-03T17:52:35.723113+0000","last_became_peered":"2024-05-03T17:52:35.723113+0000","last_unstale":"2024-05-03T17:53:19.871539+0000","last_undegraded":"2024-05-03T17:53:19.871539+0000","last_fullsized":"2024-05-03T17:53:19.871539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:37:14.606347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077441+0000","last_change":"2024-05-03T17:52:35.730858+0000","last_active":"2024-05-03T17:53:20.077441+0000","last_peered":"2024-05-03T17:53:20.077441+0000","last_clean":"2024-05-03T17:53:20.077441+0000","last_became_active":"2024-05-03T17:52:35.730453+0000","last_became_peered":"2024-05-03T17:52:35.730453+0000","last_unstale":"2024-05-03T17:53:20.077441+0000","last_undegraded":"2024-05-03T17:53:20.077441+0000","last_fullsized":"2024-05-03T17:53:20.077441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:21:17.803027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076565+0000","last_change":"2024-05-03T17:52:35.730547+0000","last_active":"2024-05-03T17:53:20.076565+0000","last_peered":"2024-05-03T17:53:20.076565+0000","last_clean":"2024-05-03T17:53:20.076565+0000","last_became_active":"2024-05-03T17:52:35.729582+0000","last_became_peered":"2024-05-03T17:52:35.729582+0000","last_unstale":"2024-05-03T17:53:20.076565+0000","last_undegraded":"2024-05-03T17:53:20.076565+0000","last_fullsized":"2024-05-03T17:53:20.076565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:24:55.110537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.868010+0000","last_change":"2024-05-03T17:52:35.726305+0000","last_active":"2024-05-03T17:53:19.868010+0000","last_peered":"2024-05-03T17:53:19.868010+0000","last_clean":"2024-05-03T17:53:19.868010+0000","last_became_active":"2024-05-03T17:52:35.725858+0000","last_became_peered":"2024-05-03T17:52:35.725858+0000","last_unstale":"2024-05-03T17:53:19.868010+0000","last_undegraded":"2024-05-03T17:53:19.868010+0000","last_fullsized":"2024-05-03T17:53:19.868010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:59.347307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871846+0000","last_change":"2024-05-03T17:52:35.726865+0000","last_active":"2024-05-03T17:53:19.871846+0000","last_peered":"2024-05-03T17:53:19.871846+0000","last_clean":"2024-05-03T17:53:19.871846+0000","last_became_active":"2024-05-03T17:52:35.726483+0000","last_became_peered":"2024-05-03T17:52:35.726483+0000","last_unstale":"2024-05-03T17:53:19.871846+0000","last_undegraded":"2024-05-03T17:53:19.871846+0000","last_fullsized":"2024-05-03T17:53:19.871846+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:17.851133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077159+0000","last_change":"2024-05-03T17:52:35.730551+0000","last_active":"2024-05-03T17:53:20.077159+0000","last_peered":"2024-05-03T17:53:20.077159+0000","last_clean":"2024-05-03T17:53:20.077159+0000","last_became_active":"2024-05-03T17:52:35.730187+0000","last_became_peered":"2024-05-03T17:52:35.730187+0000","last_unstale":"2024-05-03T17:53:20.077159+0000","last_undegraded":"2024-05-03T17:53:20.077159+0000","last_fullsized":"2024-05-03T17:53:20.077159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:18:56.211061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077071+0000","last_change":"2024-05-03T17:52:35.730040+0000","last_active":"2024-05-03T17:53:20.077071+0000","last_peered":"2024-05-03T17:53:20.077071+0000","last_clean":"2024-05-03T17:53:20.077071+0000","last_became_active":"2024-05-03T17:52:35.729549+0000","last_became_peered":"2024-05-03T17:52:35.729549+0000","last_unstale":"2024-05-03T17:53:20.077071+0000","last_undegraded":"2024-05-03T17:53:20.077071+0000","last_fullsized":"2024-05-03T17:53:20.077071+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:50:05.052247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867289+0000","last_change":"2024-05-03T17:52:35.726709+0000","last_active":"2024-05-03T17:53:19.867289+0000","last_peered":"2024-05-03T17:53:19.867289+0000","last_clean":"2024-05-03T17:53:19.867289+0000","last_became_active":"2024-05-03T17:52:35.726166+0000","last_became_peered":"2024-05-03T17:52:35.726166+0000","last_unstale":"2024-05-03T17:53:19.867289+0000","last_undegraded":"2024-05-03T17:53:19.867289+0000","last_fullsized":"2024-05-03T17:53:19.867289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:18:26.164622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870439+0000","last_change":"2024-05-03T17:52:35.727583+0000","last_active":"2024-05-03T17:53:19.870439+0000","last_peered":"2024-05-03T17:53:19.870439+0000","last_clean":"2024-05-03T17:53:19.870439+0000","last_became_active":"2024-05-03T17:52:35.727131+0000","last_became_peered":"2024-05-03T17:52:35.727131+0000","last_unstale":"2024-05-03T17:53:19.870439+0000","last_undegraded":"2024-05-03T17:53:19.870439+0000","last_fullsized":"2024-05-03T17:53:19.870439+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:28:21.255954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.887563+0000","last_change":"2024-05-03T17:53:19.887563+0000","last_active":"2024-05-03T17:53:19.887563+0000","last_peered":"2024-05-03T17:53:19.887563+0000","last_clean":"2024-05-03T17:53:19.887563+0000","last_became_active":"2024-05-03T17:53:19.887068+0000","last_became_peered":"2024-05-03T17:53:19.887068+0000","last_unstale":"2024-05-03T17:53:19.887563+0000","last_undegraded":"2024-05-03T17:53:19.887563+0000","last_fullsized":"2024-05-03T17:53:19.887563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:49:49.354138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867360+0000","last_change":"2024-05-03T17:52:35.717354+0000","last_active":"2024-05-03T17:53:19.867360+0000","last_peered":"2024-05-03T17:53:19.867360+0000","last_clean":"2024-05-03T17:53:19.867360+0000","last_became_active":"2024-05-03T17:52:35.717025+0000","last_became_peered":"2024-05-03T17:52:35.717025+0000","last_unstale":"2024-05-03T17:53:19.867360+0000","last_undegraded":"2024-05-03T17:53:19.867360+0000","last_fullsized":"2024-05-03T17:53:19.867360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:11:07.778484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870387+0000","last_change":"2024-05-03T17:52:35.725626+0000","last_active":"2024-05-03T17:53:19.870387+0000","last_peered":"2024-05-03T17:53:19.870387+0000","last_clean":"2024-05-03T17:53:19.870387+0000","last_became_active":"2024-05-03T17:52:35.725239+0000","last_became_peered":"2024-05-03T17:52:35.725239+0000","last_unstale":"2024-05-03T17:53:19.870387+0000","last_undegraded":"2024-05-03T17:53:19.870387+0000","last_fullsized":"2024-05-03T17:53:19.870387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:44:18.359484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869622+0000","last_change":"2024-05-03T17:52:35.724250+0000","last_active":"2024-05-03T17:53:19.869622+0000","last_peered":"2024-05-03T17:53:19.869622+0000","last_clean":"2024-05-03T17:53:19.869622+0000","last_became_active":"2024-05-03T17:52:35.723998+0000","last_became_peered":"2024-05-03T17:52:35.723998+0000","last_unstale":"2024-05-03T17:53:19.869622+0000","last_undegraded":"2024-05-03T17:53:19.869622+0000","last_fullsized":"2024-05-03T17:53:19.869622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:03:32.939559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886312+0000","last_change":"2024-05-03T17:52:35.726237+0000","last_active":"2024-05-03T17:53:19.886312+0000","last_peered":"2024-05-03T17:53:19.886312+0000","last_clean":"2024-05-03T17:53:19.886312+0000","last_became_active":"2024-05-03T17:52:35.725847+0000","last_became_peered":"2024-05-03T17:52:35.725847+0000","last_unstale":"2024-05-03T17:53:19.886312+0000","last_undegraded":"2024-05-03T17:53:19.886312+0000","last_fullsized":"2024-05-03T17:53:19.886312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:06.472532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869773+0000","last_change":"2024-05-03T17:52:32.687198+0000","last_active":"2024-05-03T17:53:19.869773+0000","last_peered":"2024-05-03T17:53:19.869773+0000","last_clean":"2024-05-03T17:53:19.869773+0000","last_became_active":"2024-05-03T17:52:32.686848+0000","last_became_peered":"2024-05-03T17:52:32.686848+0000","last_unstale":"2024-05-03T17:53:19.869773+0000","last_undegraded":"2024-05-03T17:53:19.869773+0000","last_fullsized":"2024-05-03T17:53:19.869773+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:08:24.244371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076791+0000","last_change":"2024-05-03T17:52:35.730595+0000","last_active":"2024-05-03T17:53:20.076791+0000","last_peered":"2024-05-03T17:53:20.076791+0000","last_clean":"2024-05-03T17:53:20.076791+0000","last_became_active":"2024-05-03T17:52:35.729999+0000","last_became_peered":"2024-05-03T17:52:35.729999+0000","last_unstale":"2024-05-03T17:53:20.076791+0000","last_undegraded":"2024-05-03T17:53:20.076791+0000","last_fullsized":"2024-05-03T17:53:20.076791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:14.485756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076952+0000","last_change":"2024-05-03T17:52:32.935239+0000","last_active":"2024-05-03T17:53:20.076952+0000","last_peered":"2024-05-03T17:53:20.076952+0000","last_clean":"2024-05-03T17:53:20.076952+0000","last_became_active":"2024-05-03T17:52:32.934915+0000","last_became_peered":"2024-05-03T17:52:32.934915+0000","last_unstale":"2024-05-03T17:53:20.076952+0000","last_undegraded":"2024-05-03T17:53:20.076952+0000","last_fullsized":"2024-05-03T17:53:20.076952+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:20.785071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870971+0000","last_change":"2024-05-03T17:52:35.724940+0000","last_active":"2024-05-03T17:53:19.870971+0000","last_peered":"2024-05-03T17:53:19.870971+0000","last_clean":"2024-05-03T17:53:19.870971+0000","last_became_active":"2024-05-03T17:52:35.724603+0000","last_became_peered":"2024-05-03T17:52:35.724603+0000","last_unstale":"2024-05-03T17:53:19.870971+0000","last_undegraded":"2024-05-03T17:53:19.870971+0000","last_fullsized":"2024-05-03T17:53:19.870971+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:43.640499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871451+0000","last_change":"2024-05-03T17:52:32.688202+0000","last_active":"2024-05-03T17:53:19.871451+0000","last_peered":"2024-05-03T17:53:19.871451+0000","last_clean":"2024-05-03T17:53:19.871451+0000","last_became_active":"2024-05-03T17:52:32.687893+0000","last_became_peered":"2024-05-03T17:52:32.687893+0000","last_unstale":"2024-05-03T17:53:19.871451+0000","last_undegraded":"2024-05-03T17:53:19.871451+0000","last_fullsized":"2024-05-03T17:53:19.871451+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:43.342832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867555+0000","last_change":"2024-05-03T17:52:35.720980+0000","last_active":"2024-05-03T17:53:19.867555+0000","last_peered":"2024-05-03T17:53:19.867555+0000","last_clean":"2024-05-03T17:53:19.867555+0000","last_became_active":"2024-05-03T17:52:35.720582+0000","last_became_peered":"2024-05-03T17:52:35.720582+0000","last_unstale":"2024-05-03T17:53:19.867555+0000","last_undegraded":"2024-05-03T17:53:19.867555+0000","last_fullsized":"2024-05-03T17:53:19.867555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:01:22.938186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.877543+0000","last_change":"2024-05-03T17:53:19.877543+0000","last_active":"2024-05-03T17:53:19.877543+0000","last_peered":"2024-05-03T17:53:19.877543+0000","last_clean":"2024-05-03T17:53:19.877543+0000","last_became_active":"2024-05-03T17:53:19.873988+0000","last_became_peered":"2024-05-03T17:53:19.873988+0000","last_unstale":"2024-05-03T17:53:19.877543+0000","last_undegraded":"2024-05-03T17:53:19.877543+0000","last_fullsized":"2024-05-03T17:53:19.877543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:49:49.105623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.079283+0000","last_change":"2024-05-03T17:53:20.078906+0000","last_active":"2024-05-03T17:53:20.079283+0000","last_peered":"2024-05-03T17:53:20.079283+0000","last_clean":"2024-05-03T17:53:20.079283+0000","last_became_active":"2024-05-03T17:53:20.078538+0000","last_became_peered":"2024-05-03T17:53:20.078538+0000","last_unstale":"2024-05-03T17:53:20.079283+0000","last_undegraded":"2024-05-03T17:53:20.079283+0000","last_fullsized":"2024-05-03T17:53:20.079283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:59:46.945417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":150,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:47.922998+0000","last_change":"2024-05-03T17:52:32.692735+0000","last_active":"2024-05-03T17:54:47.922998+0000","last_peered":"2024-05-03T17:54:47.922998+0000","last_clean":"2024-05-03T17:54:47.922998+0000","last_became_active":"2024-05-03T17:52:32.692394+0000","last_became_peered":"2024-05-03T17:52:32.692394+0000","last_unstale":"2024-05-03T17:54:47.922998+0000","last_undegraded":"2024-05-03T17:54:47.922998+0000","last_fullsized":"2024-05-03T17:54:47.922998+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:40:11.434751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870798+0000","last_change":"2024-05-03T17:52:35.727167+0000","last_active":"2024-05-03T17:53:19.870798+0000","last_peered":"2024-05-03T17:53:19.870798+0000","last_clean":"2024-05-03T17:53:19.870798+0000","last_became_active":"2024-05-03T17:52:35.726792+0000","last_became_peered":"2024-05-03T17:52:35.726792+0000","last_unstale":"2024-05-03T17:53:19.870798+0000","last_undegraded":"2024-05-03T17:53:19.870798+0000","last_fullsized":"2024-05-03T17:53:19.870798+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:54:01.823025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871452+0000","last_change":"2024-05-03T17:52:32.687573+0000","last_active":"2024-05-03T17:53:19.871452+0000","last_peered":"2024-05-03T17:53:19.871452+0000","last_clean":"2024-05-03T17:53:19.871452+0000","last_became_active":"2024-05-03T17:52:32.686158+0000","last_became_peered":"2024-05-03T17:52:32.686158+0000","last_unstale":"2024-05-03T17:53:19.871452+0000","last_undegraded":"2024-05-03T17:53:19.871452+0000","last_fullsized":"2024-05-03T17:53:19.871452+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:19.891479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873172+0000","last_change":"2024-05-03T17:52:35.728049+0000","last_active":"2024-05-03T17:53:19.873172+0000","last_peered":"2024-05-03T17:53:19.873172+0000","last_clean":"2024-05-03T17:53:19.873172+0000","last_became_active":"2024-05-03T17:52:35.727713+0000","last_became_peered":"2024-05-03T17:52:35.727713+0000","last_unstale":"2024-05-03T17:53:19.873172+0000","last_undegraded":"2024-05-03T17:53:19.873172+0000","last_fullsized":"2024-05-03T17:53:19.873172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:28:01.798457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867118+0000","last_change":"2024-05-03T17:52:32.685679+0000","last_active":"2024-05-03T17:53:19.867118+0000","last_peered":"2024-05-03T17:53:19.867118+0000","last_clean":"2024-05-03T17:53:19.867118+0000","last_became_active":"2024-05-03T17:52:32.685306+0000","last_became_peered":"2024-05-03T17:52:32.685306+0000","last_unstale":"2024-05-03T17:53:19.867118+0000","last_undegraded":"2024-05-03T17:53:19.867118+0000","last_fullsized":"2024-05-03T17:53:19.867118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:48:22.218310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076501+0000","last_change":"2024-05-03T17:52:35.717111+0000","last_active":"2024-05-03T17:53:20.076501+0000","last_peered":"2024-05-03T17:53:20.076501+0000","last_clean":"2024-05-03T17:53:20.076501+0000","last_became_active":"2024-05-03T17:52:35.716782+0000","last_became_peered":"2024-05-03T17:52:35.716782+0000","last_unstale":"2024-05-03T17:53:20.076501+0000","last_undegraded":"2024-05-03T17:53:20.076501+0000","last_fullsized":"2024-05-03T17:53:20.076501+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:31:57.766927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872477+0000","last_change":"2024-05-03T17:52:32.690846+0000","last_active":"2024-05-03T17:53:19.872477+0000","last_peered":"2024-05-03T17:53:19.872477+0000","last_clean":"2024-05-03T17:53:19.872477+0000","last_became_active":"2024-05-03T17:52:32.690506+0000","last_became_peered":"2024-05-03T17:52:32.690506+0000","last_unstale":"2024-05-03T17:53:19.872477+0000","last_undegraded":"2024-05-03T17:53:19.872477+0000","last_fullsized":"2024-05-03T17:53:19.872477+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:19.810205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869511+0000","last_change":"2024-05-03T17:52:35.718229+0000","last_active":"2024-05-03T17:53:19.869511+0000","last_peered":"2024-05-03T17:53:19.869511+0000","last_clean":"2024-05-03T17:53:19.869511+0000","last_became_active":"2024-05-03T17:52:35.717907+0000","last_became_peered":"2024-05-03T17:52:35.717907+0000","last_unstale":"2024-05-03T17:53:19.869511+0000","last_undegraded":"2024-05-03T17:53:19.869511+0000","last_fullsized":"2024-05-03T17:53:19.869511+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:45.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":143,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:45.919821+0000","last_change":"2024-05-03T17:52:32.691174+0000","last_active":"2024-05-03T17:54:45.919821+0000","last_peered":"2024-05-03T17:54:45.919821+0000","last_clean":"2024-05-03T17:54:45.919821+0000","last_became_active":"2024-05-03T17:52:32.689907+0000","last_became_peered":"2024-05-03T17:52:32.689907+0000","last_unstale":"2024-05-03T17:54:45.919821+0000","last_undegraded":"2024-05-03T17:54:45.919821+0000","last_fullsized":"2024-05-03T17:54:45.919821+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:46.630436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867946+0000","last_change":"2024-05-03T17:52:35.720290+0000","last_active":"2024-05-03T17:53:19.867946+0000","last_peered":"2024-05-03T17:53:19.867946+0000","last_clean":"2024-05-03T17:53:19.867946+0000","last_became_active":"2024-05-03T17:52:35.719962+0000","last_became_peered":"2024-05-03T17:52:35.719962+0000","last_unstale":"2024-05-03T17:53:19.867946+0000","last_undegraded":"2024-05-03T17:53:19.867946+0000","last_fullsized":"2024-05-03T17:53:19.867946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:31:41.301592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077710+0000","last_change":"2024-05-03T17:52:32.689937+0000","last_active":"2024-05-03T17:53:20.077710+0000","last_peered":"2024-05-03T17:53:20.077710+0000","last_clean":"2024-05-03T17:53:20.077710+0000","last_became_active":"2024-05-03T17:52:32.689624+0000","last_became_peered":"2024-05-03T17:52:32.689624+0000","last_unstale":"2024-05-03T17:53:20.077710+0000","last_undegraded":"2024-05-03T17:53:20.077710+0000","last_fullsized":"2024-05-03T17:53:20.077710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:56:24.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870881+0000","last_change":"2024-05-03T17:52:35.726824+0000","last_active":"2024-05-03T17:53:19.870881+0000","last_peered":"2024-05-03T17:53:19.870881+0000","last_clean":"2024-05-03T17:53:19.870881+0000","last_became_active":"2024-05-03T17:52:35.726494+0000","last_became_peered":"2024-05-03T17:52:35.726494+0000","last_unstale":"2024-05-03T17:53:19.870881+0000","last_undegraded":"2024-05-03T17:53:19.870881+0000","last_fullsized":"2024-05-03T17:53:19.870881+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:41:21.284852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873051+0000","last_change":"2024-05-03T17:52:32.935471+0000","last_active":"2024-05-03T17:53:19.873051+0000","last_peered":"2024-05-03T17:53:19.873051+0000","last_clean":"2024-05-03T17:53:19.873051+0000","last_became_active":"2024-05-03T17:52:32.935012+0000","last_became_peered":"2024-05-03T17:52:32.935012+0000","last_unstale":"2024-05-03T17:53:19.873051+0000","last_undegraded":"2024-05-03T17:53:19.873051+0000","last_fullsized":"2024-05-03T17:53:19.873051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:35:38.654577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867634+0000","last_change":"2024-05-03T17:52:35.727124+0000","last_active":"2024-05-03T17:53:19.867634+0000","last_peered":"2024-05-03T17:53:19.867634+0000","last_clean":"2024-05-03T17:53:19.867634+0000","last_became_active":"2024-05-03T17:52:35.726804+0000","last_became_peered":"2024-05-03T17:52:35.726804+0000","last_unstale":"2024-05-03T17:53:19.867634+0000","last_undegraded":"2024-05-03T17:53:19.867634+0000","last_fullsized":"2024-05-03T17:53:19.867634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:32.000803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'29","reported_seq":292,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:46.921139+0000","last_change":"2024-05-03T17:52:32.685014+0000","last_active":"2024-05-03T17:54:46.921139+0000","last_peered":"2024-05-03T17:54:46.921139+0000","last_clean":"2024-05-03T17:54:46.921139+0000","last_became_active":"2024-05-03T17:52:32.684660+0000","last_became_peered":"2024-05-03T17:52:32.684660+0000","last_unstale":"2024-05-03T17:54:46.921139+0000","last_undegraded":"2024-05-03T17:54:46.921139+0000","last_fullsized":"2024-05-03T17:54:46.921139+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:25:56.890283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54817,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":237,"num_read_kb":171,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871281+0000","last_change":"2024-05-03T17:52:35.733675+0000","last_active":"2024-05-03T17:53:19.871281+0000","last_peered":"2024-05-03T17:53:19.871281+0000","last_clean":"2024-05-03T17:53:19.871281+0000","last_became_active":"2024-05-03T17:52:35.733234+0000","last_became_peered":"2024-05-03T17:52:35.733234+0000","last_unstale":"2024-05-03T17:53:19.871281+0000","last_undegraded":"2024-05-03T17:53:19.871281+0000","last_fullsized":"2024-05-03T17:53:19.871281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:57.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886182+0000","last_change":"2024-05-03T17:52:32.690462+0000","last_active":"2024-05-03T17:53:19.886182+0000","last_peered":"2024-05-03T17:53:19.886182+0000","last_clean":"2024-05-03T17:53:19.886182+0000","last_became_active":"2024-05-03T17:52:32.690169+0000","last_became_peered":"2024-05-03T17:52:32.690169+0000","last_unstale":"2024-05-03T17:53:19.886182+0000","last_undegraded":"2024-05-03T17:53:19.886182+0000","last_fullsized":"2024-05-03T17:53:19.886182+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:59.324612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867010+0000","last_change":"2024-05-03T17:52:35.720730+0000","last_active":"2024-05-03T17:53:19.867010+0000","last_peered":"2024-05-03T17:53:19.867010+0000","last_clean":"2024-05-03T17:53:19.867010+0000","last_became_active":"2024-05-03T17:52:35.720392+0000","last_became_peered":"2024-05-03T17:52:35.720392+0000","last_unstale":"2024-05-03T17:53:19.867010+0000","last_undegraded":"2024-05-03T17:53:19.867010+0000","last_fullsized":"2024-05-03T17:53:19.867010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:58.528148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872742+0000","last_change":"2024-05-03T17:52:32.691439+0000","last_active":"2024-05-03T17:53:19.872742+0000","last_peered":"2024-05-03T17:53:19.872742+0000","last_clean":"2024-05-03T17:53:19.872742+0000","last_became_active":"2024-05-03T17:52:32.691080+0000","last_became_peered":"2024-05-03T17:52:32.691080+0000","last_unstale":"2024-05-03T17:53:19.872742+0000","last_undegraded":"2024-05-03T17:53:19.872742+0000","last_fullsized":"2024-05-03T17:53:19.872742+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:40:43.120194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077160+0000","last_change":"2024-05-03T17:52:35.729860+0000","last_active":"2024-05-03T17:53:20.077160+0000","last_peered":"2024-05-03T17:53:20.077160+0000","last_clean":"2024-05-03T17:53:20.077160+0000","last_became_active":"2024-05-03T17:52:35.729508+0000","last_became_peered":"2024-05-03T17:52:35.729508+0000","last_unstale":"2024-05-03T17:53:20.077160+0000","last_undegraded":"2024-05-03T17:53:20.077160+0000","last_fullsized":"2024-05-03T17:53:20.077160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:39:20.859215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":149,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:47.922702+0000","last_change":"2024-05-03T17:52:32.689346+0000","last_active":"2024-05-03T17:54:47.922702+0000","last_peered":"2024-05-03T17:54:47.922702+0000","last_clean":"2024-05-03T17:54:47.922702+0000","last_became_active":"2024-05-03T17:52:32.689009+0000","last_became_peered":"2024-05-03T17:52:32.689009+0000","last_unstale":"2024-05-03T17:54:47.922702+0000","last_undegraded":"2024-05-03T17:54:47.922702+0000","last_fullsized":"2024-05-03T17:54:47.922702+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:55:36.704929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077075+0000","last_change":"2024-05-03T17:52:35.729226+0000","last_active":"2024-05-03T17:53:20.077075+0000","last_peered":"2024-05-03T17:53:20.077075+0000","last_clean":"2024-05-03T17:53:20.077075+0000","last_became_active":"2024-05-03T17:52:35.728846+0000","last_became_peered":"2024-05-03T17:52:35.728846+0000","last_unstale":"2024-05-03T17:53:20.077075+0000","last_undegraded":"2024-05-03T17:53:20.077075+0000","last_fullsized":"2024-05-03T17:53:20.077075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:39:16.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870410+0000","last_change":"2024-05-03T17:52:32.936108+0000","last_active":"2024-05-03T17:53:19.870410+0000","last_peered":"2024-05-03T17:53:19.870410+0000","last_clean":"2024-05-03T17:53:19.870410+0000","last_became_active":"2024-05-03T17:52:32.935782+0000","last_became_peered":"2024-05-03T17:52:32.935782+0000","last_unstale":"2024-05-03T17:53:19.870410+0000","last_undegraded":"2024-05-03T17:53:19.870410+0000","last_fullsized":"2024-05-03T17:53:19.870410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:47.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872695+0000","last_change":"2024-05-03T17:52:35.727163+0000","last_active":"2024-05-03T17:53:19.872695+0000","last_peered":"2024-05-03T17:53:19.872695+0000","last_clean":"2024-05-03T17:53:19.872695+0000","last_became_active":"2024-05-03T17:52:35.726824+0000","last_became_peered":"2024-05-03T17:52:35.726824+0000","last_unstale":"2024-05-03T17:53:19.872695+0000","last_undegraded":"2024-05-03T17:53:19.872695+0000","last_fullsized":"2024-05-03T17:53:19.872695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:24:50.803860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871601+0000","last_change":"2024-05-03T17:52:32.687329+0000","last_active":"2024-05-03T17:53:19.871601+0000","last_peered":"2024-05-03T17:53:19.871601+0000","last_clean":"2024-05-03T17:53:19.871601+0000","last_became_active":"2024-05-03T17:52:32.687025+0000","last_became_peered":"2024-05-03T17:52:32.687025+0000","last_unstale":"2024-05-03T17:53:19.871601+0000","last_undegraded":"2024-05-03T17:53:19.871601+0000","last_fullsized":"2024-05-03T17:53:19.871601+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:42:15.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869955+0000","last_change":"2024-05-03T17:52:35.728797+0000","last_active":"2024-05-03T17:53:19.869955+0000","last_peered":"2024-05-03T17:53:19.869955+0000","last_clean":"2024-05-03T17:53:19.869955+0000","last_became_active":"2024-05-03T17:52:35.728429+0000","last_became_peered":"2024-05-03T17:52:35.728429+0000","last_unstale":"2024-05-03T17:53:19.869955+0000","last_undegraded":"2024-05-03T17:53:19.869955+0000","last_fullsized":"2024-05-03T17:53:19.869955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:19.584593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872992+0000","last_change":"2024-05-03T17:53:19.872508+0000","last_active":"2024-05-03T17:53:19.872992+0000","last_peered":"2024-05-03T17:53:19.872992+0000","last_clean":"2024-05-03T17:53:19.872992+0000","last_became_active":"2024-05-03T17:53:19.872175+0000","last_became_peered":"2024-05-03T17:53:19.872175+0000","last_unstale":"2024-05-03T17:53:19.872992+0000","last_undegraded":"2024-05-03T17:53:19.872992+0000","last_fullsized":"2024-05-03T17:53:19.872992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:24:08.246884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871648+0000","last_change":"2024-05-03T17:52:35.719115+0000","last_active":"2024-05-03T17:53:19.871648+0000","last_peered":"2024-05-03T17:53:19.871648+0000","last_clean":"2024-05-03T17:53:19.871648+0000","last_became_active":"2024-05-03T17:52:35.718054+0000","last_became_peered":"2024-05-03T17:52:35.718054+0000","last_unstale":"2024-05-03T17:53:19.871648+0000","last_undegraded":"2024-05-03T17:53:19.871648+0000","last_fullsized":"2024-05-03T17:53:19.871648+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.903766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077630+0000","last_change":"2024-05-03T17:52:32.688993+0000","last_active":"2024-05-03T17:53:20.077630+0000","last_peered":"2024-05-03T17:53:20.077630+0000","last_clean":"2024-05-03T17:53:20.077630+0000","last_became_active":"2024-05-03T17:52:32.688706+0000","last_became_peered":"2024-05-03T17:52:32.688706+0000","last_unstale":"2024-05-03T17:53:20.077630+0000","last_undegraded":"2024-05-03T17:53:20.077630+0000","last_fullsized":"2024-05-03T17:53:20.077630+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:40:14.023338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873314+0000","last_change":"2024-05-03T17:53:19.872957+0000","last_active":"2024-05-03T17:53:19.873314+0000","last_peered":"2024-05-03T17:53:19.873314+0000","last_clean":"2024-05-03T17:53:19.873314+0000","last_became_active":"2024-05-03T17:53:19.872690+0000","last_became_peered":"2024-05-03T17:53:19.872690+0000","last_unstale":"2024-05-03T17:53:19.873314+0000","last_undegraded":"2024-05-03T17:53:19.873314+0000","last_fullsized":"2024-05-03T17:53:19.873314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:53:43.801983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.873043+0000","last_change":"2024-05-03T17:52:35.728958+0000","last_active":"2024-05-03T17:53:19.873043+0000","last_peered":"2024-05-03T17:53:19.873043+0000","last_clean":"2024-05-03T17:53:19.873043+0000","last_became_active":"2024-05-03T17:52:35.728619+0000","last_became_peered":"2024-05-03T17:52:35.728619+0000","last_unstale":"2024-05-03T17:53:19.873043+0000","last_undegraded":"2024-05-03T17:53:19.873043+0000","last_fullsized":"2024-05-03T17:53:19.873043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:29:29.922324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":146,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:46.920879+0000","last_change":"2024-05-03T17:52:32.935367+0000","last_active":"2024-05-03T17:54:46.920879+0000","last_peered":"2024-05-03T17:54:46.920879+0000","last_clean":"2024-05-03T17:54:46.920879+0000","last_became_active":"2024-05-03T17:52:32.934938+0000","last_became_peered":"2024-05-03T17:52:32.934938+0000","last_unstale":"2024-05-03T17:54:46.920879+0000","last_undegraded":"2024-05-03T17:54:46.920879+0000","last_fullsized":"2024-05-03T17:54:46.920879+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:07:21.642883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076712+0000","last_change":"2024-05-03T17:52:35.731035+0000","last_active":"2024-05-03T17:53:20.076712+0000","last_peered":"2024-05-03T17:53:20.076712+0000","last_clean":"2024-05-03T17:53:20.076712+0000","last_became_active":"2024-05-03T17:52:35.730034+0000","last_became_peered":"2024-05-03T17:52:35.730034+0000","last_unstale":"2024-05-03T17:53:20.076712+0000","last_undegraded":"2024-05-03T17:53:20.076712+0000","last_fullsized":"2024-05-03T17:53:20.076712+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:25.545290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077931+0000","last_change":"2024-05-03T17:52:32.690043+0000","last_active":"2024-05-03T17:53:20.077931+0000","last_peered":"2024-05-03T17:53:20.077931+0000","last_clean":"2024-05-03T17:53:20.077931+0000","last_became_active":"2024-05-03T17:52:32.689562+0000","last_became_peered":"2024-05-03T17:52:32.689562+0000","last_unstale":"2024-05-03T17:53:20.077931+0000","last_undegraded":"2024-05-03T17:53:20.077931+0000","last_fullsized":"2024-05-03T17:53:20.077931+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:59.014968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077348+0000","last_change":"2024-05-03T17:51:02.340854+0000","last_active":"2024-05-03T17:53:20.077348+0000","last_peered":"2024-05-03T17:53:20.077348+0000","last_clean":"2024-05-03T17:53:20.077348+0000","last_became_active":"2024-05-03T17:51:02.336454+0000","last_became_peered":"2024-05-03T17:51:02.336454+0000","last_unstale":"2024-05-03T17:53:20.077348+0000","last_undegraded":"2024-05-03T17:53:20.077348+0000","last_fullsized":"2024-05-03T17:53:20.077348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:47:41.306771+0000","last_clean_scrub_stamp":"2024-05-03T17:47:41.306771+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:18:08.512098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.887660+0000","last_change":"2024-05-03T17:53:19.887126+0000","last_active":"2024-05-03T17:53:19.887660+0000","last_peered":"2024-05-03T17:53:19.887660+0000","last_clean":"2024-05-03T17:53:19.887660+0000","last_became_active":"2024-05-03T17:53:19.886736+0000","last_became_peered":"2024-05-03T17:53:19.886736+0000","last_unstale":"2024-05-03T17:53:19.887660+0000","last_undegraded":"2024-05-03T17:53:19.887660+0000","last_fullsized":"2024-05-03T17:53:19.887660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:35:02.036825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077451+0000","last_change":"2024-05-03T17:52:32.690093+0000","last_active":"2024-05-03T17:53:20.077451+0000","last_peered":"2024-05-03T17:53:20.077451+0000","last_clean":"2024-05-03T17:53:20.077451+0000","last_became_active":"2024-05-03T17:52:32.689777+0000","last_became_peered":"2024-05-03T17:52:32.689777+0000","last_unstale":"2024-05-03T17:53:20.077451+0000","last_undegraded":"2024-05-03T17:53:20.077451+0000","last_fullsized":"2024-05-03T17:53:20.077451+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:09.693011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.871391+0000","last_change":"2024-05-03T17:52:32.935589+0000","last_active":"2024-05-03T17:53:19.871391+0000","last_peered":"2024-05-03T17:53:19.871391+0000","last_clean":"2024-05-03T17:53:19.871391+0000","last_became_active":"2024-05-03T17:52:32.935221+0000","last_became_peered":"2024-05-03T17:52:32.935221+0000","last_unstale":"2024-05-03T17:53:19.871391+0000","last_undegraded":"2024-05-03T17:53:19.871391+0000","last_fullsized":"2024-05-03T17:53:19.871391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:26:08.001761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867238+0000","last_change":"2024-05-03T17:52:35.724293+0000","last_active":"2024-05-03T17:53:19.867238+0000","last_peered":"2024-05-03T17:53:19.867238+0000","last_clean":"2024-05-03T17:53:19.867238+0000","last_became_active":"2024-05-03T17:52:35.723978+0000","last_became_peered":"2024-05-03T17:52:35.723978+0000","last_unstale":"2024-05-03T17:53:19.867238+0000","last_undegraded":"2024-05-03T17:53:19.867238+0000","last_fullsized":"2024-05-03T17:53:19.867238+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:23.357872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'10","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:54:19.733474+0000","last_change":"2024-05-03T17:52:32.687303+0000","last_active":"2024-05-03T17:54:19.733474+0000","last_peered":"2024-05-03T17:54:19.733474+0000","last_clean":"2024-05-03T17:54:19.733474+0000","last_became_active":"2024-05-03T17:52:32.686827+0000","last_became_peered":"2024-05-03T17:52:32.686827+0000","last_unstale":"2024-05-03T17:54:19.733474+0000","last_undegraded":"2024-05-03T17:54:19.733474+0000","last_fullsized":"2024-05-03T17:54:19.733474+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:11:53.565215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076907+0000","last_change":"2024-05-03T17:52:35.720898+0000","last_active":"2024-05-03T17:53:20.076907+0000","last_peered":"2024-05-03T17:53:20.076907+0000","last_clean":"2024-05-03T17:53:20.076907+0000","last_became_active":"2024-05-03T17:52:35.720572+0000","last_became_peered":"2024-05-03T17:52:35.720572+0000","last_unstale":"2024-05-03T17:53:20.076907+0000","last_undegraded":"2024-05-03T17:53:20.076907+0000","last_fullsized":"2024-05-03T17:53:20.076907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:01.697167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077417+0000","last_change":"2024-05-03T17:52:33.249970+0000","last_active":"2024-05-03T17:53:20.077417+0000","last_peered":"2024-05-03T17:53:20.077417+0000","last_clean":"2024-05-03T17:53:20.077417+0000","last_became_active":"2024-05-03T17:52:33.249596+0000","last_became_peered":"2024-05-03T17:52:33.249596+0000","last_unstale":"2024-05-03T17:53:20.077417+0000","last_undegraded":"2024-05-03T17:53:20.077417+0000","last_fullsized":"2024-05-03T17:53:20.077417+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:46:53.058586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870889+0000","last_change":"2024-05-03T17:52:35.722129+0000","last_active":"2024-05-03T17:53:19.870889+0000","last_peered":"2024-05-03T17:53:19.870889+0000","last_clean":"2024-05-03T17:53:19.870889+0000","last_became_active":"2024-05-03T17:52:35.721815+0000","last_became_peered":"2024-05-03T17:52:35.721815+0000","last_unstale":"2024-05-03T17:53:19.870889+0000","last_undegraded":"2024-05-03T17:53:19.870889+0000","last_fullsized":"2024-05-03T17:53:19.870889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:33.781350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867452+0000","last_change":"2024-05-03T17:52:32.936157+0000","last_active":"2024-05-03T17:53:19.867452+0000","last_peered":"2024-05-03T17:53:19.867452+0000","last_clean":"2024-05-03T17:53:19.867452+0000","last_became_active":"2024-05-03T17:52:32.935842+0000","last_became_peered":"2024-05-03T17:52:32.935842+0000","last_unstale":"2024-05-03T17:53:19.867452+0000","last_undegraded":"2024-05-03T17:53:19.867452+0000","last_fullsized":"2024-05-03T17:53:19.867452+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:53:18.006451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870559+0000","last_change":"2024-05-03T17:52:35.735993+0000","last_active":"2024-05-03T17:53:19.870559+0000","last_peered":"2024-05-03T17:53:19.870559+0000","last_clean":"2024-05-03T17:53:19.870559+0000","last_became_active":"2024-05-03T17:52:35.735612+0000","last_became_peered":"2024-05-03T17:52:35.735612+0000","last_unstale":"2024-05-03T17:53:19.870559+0000","last_undegraded":"2024-05-03T17:53:19.870559+0000","last_fullsized":"2024-05-03T17:53:19.870559+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:59.216022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076687+0000","last_change":"2024-05-03T17:52:32.687147+0000","last_active":"2024-05-03T17:53:20.076687+0000","last_peered":"2024-05-03T17:53:20.076687+0000","last_clean":"2024-05-03T17:53:20.076687+0000","last_became_active":"2024-05-03T17:52:32.686831+0000","last_became_peered":"2024-05-03T17:52:32.686831+0000","last_unstale":"2024-05-03T17:53:20.076687+0000","last_undegraded":"2024-05-03T17:53:20.076687+0000","last_fullsized":"2024-05-03T17:53:20.076687+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:24:30.092829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869744+0000","last_change":"2024-05-03T17:52:35.721862+0000","last_active":"2024-05-03T17:53:19.869744+0000","last_peered":"2024-05-03T17:53:19.869744+0000","last_clean":"2024-05-03T17:53:19.869744+0000","last_became_active":"2024-05-03T17:52:35.721393+0000","last_became_peered":"2024-05-03T17:52:35.721393+0000","last_unstale":"2024-05-03T17:53:19.869744+0000","last_undegraded":"2024-05-03T17:53:19.869744+0000","last_fullsized":"2024-05-03T17:53:19.869744+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:47:32.014817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886007+0000","last_change":"2024-05-03T17:52:32.687867+0000","last_active":"2024-05-03T17:53:19.886007+0000","last_peered":"2024-05-03T17:53:19.886007+0000","last_clean":"2024-05-03T17:53:19.886007+0000","last_became_active":"2024-05-03T17:52:32.687557+0000","last_became_peered":"2024-05-03T17:52:32.687557+0000","last_unstale":"2024-05-03T17:53:19.886007+0000","last_undegraded":"2024-05-03T17:53:19.886007+0000","last_fullsized":"2024-05-03T17:53:19.886007+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:45.863180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076840+0000","last_change":"2024-05-03T17:52:35.731057+0000","last_active":"2024-05-03T17:53:20.076840+0000","last_peered":"2024-05-03T17:53:20.076840+0000","last_clean":"2024-05-03T17:53:20.076840+0000","last_became_active":"2024-05-03T17:52:35.729293+0000","last_became_peered":"2024-05-03T17:52:35.729293+0000","last_unstale":"2024-05-03T17:53:20.076840+0000","last_undegraded":"2024-05-03T17:53:20.076840+0000","last_fullsized":"2024-05-03T17:53:20.076840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:12:53.888078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870778+0000","last_change":"2024-05-03T17:52:32.689675+0000","last_active":"2024-05-03T17:53:19.870778+0000","last_peered":"2024-05-03T17:53:19.870778+0000","last_clean":"2024-05-03T17:53:19.870778+0000","last_became_active":"2024-05-03T17:52:32.689340+0000","last_became_peered":"2024-05-03T17:52:32.689340+0000","last_unstale":"2024-05-03T17:53:19.870778+0000","last_undegraded":"2024-05-03T17:53:19.870778+0000","last_fullsized":"2024-05-03T17:53:19.870778+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:31.398045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077645+0000","last_change":"2024-05-03T17:52:35.728228+0000","last_active":"2024-05-03T17:53:20.077645+0000","last_peered":"2024-05-03T17:53:20.077645+0000","last_clean":"2024-05-03T17:53:20.077645+0000","last_became_active":"2024-05-03T17:52:35.727233+0000","last_became_peered":"2024-05-03T17:52:35.727233+0000","last_unstale":"2024-05-03T17:53:20.077645+0000","last_undegraded":"2024-05-03T17:53:20.077645+0000","last_fullsized":"2024-05-03T17:53:20.077645+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:57.153154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886474+0000","last_change":"2024-05-03T17:52:33.249690+0000","last_active":"2024-05-03T17:53:19.886474+0000","last_peered":"2024-05-03T17:53:19.886474+0000","last_clean":"2024-05-03T17:53:19.886474+0000","last_became_active":"2024-05-03T17:52:33.249302+0000","last_became_peered":"2024-05-03T17:52:33.249302+0000","last_unstale":"2024-05-03T17:53:19.886474+0000","last_undegraded":"2024-05-03T17:53:19.886474+0000","last_fullsized":"2024-05-03T17:53:19.886474+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:05:48.700670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077737+0000","last_change":"2024-05-03T17:52:35.727171+0000","last_active":"2024-05-03T17:53:20.077737+0000","last_peered":"2024-05-03T17:53:20.077737+0000","last_clean":"2024-05-03T17:53:20.077737+0000","last_became_active":"2024-05-03T17:52:35.726834+0000","last_became_peered":"2024-05-03T17:52:35.726834+0000","last_unstale":"2024-05-03T17:53:20.077737+0000","last_undegraded":"2024-05-03T17:53:20.077737+0000","last_fullsized":"2024-05-03T17:53:20.077737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:40:09.938183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872852+0000","last_change":"2024-05-03T17:52:32.693694+0000","last_active":"2024-05-03T17:53:19.872852+0000","last_peered":"2024-05-03T17:53:19.872852+0000","last_clean":"2024-05-03T17:53:19.872852+0000","last_became_active":"2024-05-03T17:52:32.693310+0000","last_became_peered":"2024-05-03T17:52:32.693310+0000","last_unstale":"2024-05-03T17:53:19.872852+0000","last_undegraded":"2024-05-03T17:53:19.872852+0000","last_fullsized":"2024-05-03T17:53:19.872852+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:59:37.449434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077538+0000","last_change":"2024-05-03T17:52:35.728267+0000","last_active":"2024-05-03T17:53:20.077538+0000","last_peered":"2024-05-03T17:53:20.077538+0000","last_clean":"2024-05-03T17:53:20.077538+0000","last_became_active":"2024-05-03T17:52:35.727902+0000","last_became_peered":"2024-05-03T17:52:35.727902+0000","last_unstale":"2024-05-03T17:53:20.077538+0000","last_undegraded":"2024-05-03T17:53:20.077538+0000","last_fullsized":"2024-05-03T17:53:20.077538+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:09.223850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867198+0000","last_change":"2024-05-03T17:52:32.687248+0000","last_active":"2024-05-03T17:53:19.867198+0000","last_peered":"2024-05-03T17:53:19.867198+0000","last_clean":"2024-05-03T17:53:19.867198+0000","last_became_active":"2024-05-03T17:52:32.686921+0000","last_became_peered":"2024-05-03T17:52:32.686921+0000","last_unstale":"2024-05-03T17:53:19.867198+0000","last_undegraded":"2024-05-03T17:53:19.867198+0000","last_fullsized":"2024-05-03T17:53:19.867198+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:31.671599+0000","last_clean_scrub_stamp":"2024-05-03T17:52:31.671599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:35:40.849484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.077858+0000","last_change":"2024-05-03T17:52:35.729383+0000","last_active":"2024-05-03T17:53:20.077858+0000","last_peered":"2024-05-03T17:53:20.077858+0000","last_clean":"2024-05-03T17:53:20.077858+0000","last_became_active":"2024-05-03T17:52:35.729037+0000","last_became_peered":"2024-05-03T17:52:35.729037+0000","last_unstale":"2024-05-03T17:53:20.077858+0000","last_undegraded":"2024-05-03T17:53:20.077858+0000","last_fullsized":"2024-05-03T17:53:20.077858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:03:03.815770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.869889+0000","last_change":"2024-05-03T17:52:35.731809+0000","last_active":"2024-05-03T17:53:19.869889+0000","last_peered":"2024-05-03T17:53:19.869889+0000","last_clean":"2024-05-03T17:53:19.869889+0000","last_became_active":"2024-05-03T17:52:35.729360+0000","last_became_peered":"2024-05-03T17:52:35.729360+0000","last_unstale":"2024-05-03T17:53:19.869889+0000","last_undegraded":"2024-05-03T17:53:19.869889+0000","last_fullsized":"2024-05-03T17:53:19.869889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:03:22.181184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886214+0000","last_change":"2024-05-03T17:52:35.727771+0000","last_active":"2024-05-03T17:53:19.886214+0000","last_peered":"2024-05-03T17:53:19.886214+0000","last_clean":"2024-05-03T17:53:19.886214+0000","last_became_active":"2024-05-03T17:52:35.727414+0000","last_became_peered":"2024-05-03T17:52:35.727414+0000","last_unstale":"2024-05-03T17:53:19.886214+0000","last_undegraded":"2024-05-03T17:53:19.886214+0000","last_fullsized":"2024-05-03T17:53:19.886214+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:09:02.905481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.885945+0000","last_change":"2024-05-03T17:52:35.727433+0000","last_active":"2024-05-03T17:53:19.885945+0000","last_peered":"2024-05-03T17:53:19.885945+0000","last_clean":"2024-05-03T17:53:19.885945+0000","last_became_active":"2024-05-03T17:52:35.727123+0000","last_became_peered":"2024-05-03T17:52:35.727123+0000","last_unstale":"2024-05-03T17:53:19.885945+0000","last_undegraded":"2024-05-03T17:53:19.885945+0000","last_fullsized":"2024-05-03T17:53:19.885945+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:41:36.871788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870777+0000","last_change":"2024-05-03T17:52:35.721332+0000","last_active":"2024-05-03T17:53:19.870777+0000","last_peered":"2024-05-03T17:53:19.870777+0000","last_clean":"2024-05-03T17:53:19.870777+0000","last_became_active":"2024-05-03T17:52:35.721004+0000","last_became_peered":"2024-05-03T17:52:35.721004+0000","last_unstale":"2024-05-03T17:53:19.870777+0000","last_undegraded":"2024-05-03T17:53:19.870777+0000","last_fullsized":"2024-05-03T17:53:19.870777+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:35:03.443163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867358+0000","last_change":"2024-05-03T17:52:35.723649+0000","last_active":"2024-05-03T17:53:19.867358+0000","last_peered":"2024-05-03T17:53:19.867358+0000","last_clean":"2024-05-03T17:53:19.867358+0000","last_became_active":"2024-05-03T17:52:35.723303+0000","last_became_peered":"2024-05-03T17:52:35.723303+0000","last_unstale":"2024-05-03T17:53:19.867358+0000","last_undegraded":"2024-05-03T17:53:19.867358+0000","last_fullsized":"2024-05-03T17:53:19.867358+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:02:49.093039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870148+0000","last_change":"2024-05-03T17:52:35.732505+0000","last_active":"2024-05-03T17:53:19.870148+0000","last_peered":"2024-05-03T17:53:19.870148+0000","last_clean":"2024-05-03T17:53:19.870148+0000","last_became_active":"2024-05-03T17:52:35.732180+0000","last_became_peered":"2024-05-03T17:52:35.732180+0000","last_unstale":"2024-05-03T17:53:19.870148+0000","last_undegraded":"2024-05-03T17:53:19.870148+0000","last_fullsized":"2024-05-03T17:53:19.870148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:32:54.371260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.886392+0000","last_change":"2024-05-03T17:52:35.732217+0000","last_active":"2024-05-03T17:53:19.886392+0000","last_peered":"2024-05-03T17:53:19.886392+0000","last_clean":"2024-05-03T17:53:19.886392+0000","last_became_active":"2024-05-03T17:52:35.731868+0000","last_became_peered":"2024-05-03T17:52:35.731868+0000","last_unstale":"2024-05-03T17:53:19.886392+0000","last_undegraded":"2024-05-03T17:53:19.886392+0000","last_fullsized":"2024-05-03T17:53:19.886392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:04:17.521528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076863+0000","last_change":"2024-05-03T17:52:35.723641+0000","last_active":"2024-05-03T17:53:20.076863+0000","last_peered":"2024-05-03T17:53:20.076863+0000","last_clean":"2024-05-03T17:53:20.076863+0000","last_became_active":"2024-05-03T17:52:35.723323+0000","last_became_peered":"2024-05-03T17:52:35.723323+0000","last_unstale":"2024-05-03T17:53:20.076863+0000","last_undegraded":"2024-05-03T17:53:20.076863+0000","last_fullsized":"2024-05-03T17:53:20.076863+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:45:34.032407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.872534+0000","last_change":"2024-05-03T17:52:35.731008+0000","last_active":"2024-05-03T17:53:19.872534+0000","last_peered":"2024-05-03T17:53:19.872534+0000","last_clean":"2024-05-03T17:53:19.872534+0000","last_became_active":"2024-05-03T17:52:35.730627+0000","last_became_peered":"2024-05-03T17:52:35.730627+0000","last_unstale":"2024-05-03T17:53:19.872534+0000","last_undegraded":"2024-05-03T17:53:19.872534+0000","last_fullsized":"2024-05-03T17:53:19.872534+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:30:32.884518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:20.076519+0000","last_change":"2024-05-03T17:52:35.726938+0000","last_active":"2024-05-03T17:53:20.076519+0000","last_peered":"2024-05-03T17:53:20.076519+0000","last_clean":"2024-05-03T17:53:20.076519+0000","last_became_active":"2024-05-03T17:52:35.726611+0000","last_became_peered":"2024-05-03T17:52:35.726611+0000","last_unstale":"2024-05-03T17:53:20.076519+0000","last_undegraded":"2024-05-03T17:53:20.076519+0000","last_fullsized":"2024-05-03T17:53:20.076519+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:11.587035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867632+0000","last_change":"2024-05-03T17:52:35.722562+0000","last_active":"2024-05-03T17:53:19.867632+0000","last_peered":"2024-05-03T17:53:19.867632+0000","last_clean":"2024-05-03T17:53:19.867632+0000","last_became_active":"2024-05-03T17:52:35.722229+0000","last_became_peered":"2024-05-03T17:52:35.722229+0000","last_unstale":"2024-05-03T17:53:19.867632+0000","last_undegraded":"2024-05-03T17:53:19.867632+0000","last_fullsized":"2024-05-03T17:53:19.867632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:57:06.730533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.867188+0000","last_change":"2024-05-03T17:52:35.722707+0000","last_active":"2024-05-03T17:53:19.867188+0000","last_peered":"2024-05-03T17:53:19.867188+0000","last_clean":"2024-05-03T17:53:19.867188+0000","last_became_active":"2024-05-03T17:52:35.721847+0000","last_became_peered":"2024-05-03T17:52:35.721847+0000","last_unstale":"2024-05-03T17:53:19.867188+0000","last_undegraded":"2024-05-03T17:53:19.867188+0000","last_fullsized":"2024-05-03T17:53:19.867188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:56:21.620074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-03T17:53:19.870547+0000","last_change":"2024-05-03T17:52:35.731810+0000","last_active":"2024-05-03T17:53:19.870547+0000","last_peered":"2024-05-03T17:53:19.870547+0000","last_clean":"2024-05-03T17:53:19.870547+0000","last_became_active":"2024-05-03T17:52:35.728986+0000","last_became_peered":"2024-05-03T17:52:35.728986+0000","last_unstale":"2024-05-03T17:53:19.870547+0000","last_undegraded":"2024-05-03T17:53:19.870547+0000","last_fullsized":"2024-05-03T17:53:19.870547+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T17:52:34.691304+0000","last_clean_scrub_stamp":"2024-05-03T17:52:34.691304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:18.275792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280811,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59209,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":718,"num_read_kb":420,"num_write":164,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":177627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743468,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":417972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906993,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1064661,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070518,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":418208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266747,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":418128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462976,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":1010449,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626501,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":417597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":913408,"data_stored":472207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":418868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":473948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":872448,"data_stored":417908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986256,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":860160,"data_stored":418425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822731,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":1009742,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35710,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T17:54:51.874 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-03T17:54:51.875 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-03T17:54:51.875 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-03T17:54:51.875 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-03T17:54:52.113 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:52.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:52 smithi122 ceph-mon[29744]: pgmap v355: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:52.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:52 smithi122 ceph-mon[29744]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:54:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:52 smithi192 ceph-mon[33753]: pgmap v355: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:52 smithi192 ceph-mon[33753]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:54:52.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:52 smithi149 ceph-mon[33409]: pgmap v355: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:54:52.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:52 smithi149 ceph-mon[33409]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T17:54:53.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.812+0000 7f11f52ed700 1 -- 172.21.15.122:0/3310957697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11f0102820 msgr2=0x7f11f0110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:53.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.812+0000 7f11f52ed700 1 --2- 172.21.15.122:0/3310957697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11f0102820 0x7f11f0110290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f11e800b0a0 tx=0x7f11e800b3b0 comp rx=0 tx=0).stop 2024-05-03T17:54:53.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.812+0000 7f11f52ed700 1 -- 172.21.15.122:0/3310957697 shutdown_connections 2024-05-03T17:54:53.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.812+0000 7f11f52ed700 1 --2- 172.21.15.122:0/3310957697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11f0102820 0x7f11f0110290 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:53.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.812+0000 7f11f52ed700 1 --2- 172.21.15.122:0/3310957697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11f0108580 0x7f11f0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:53.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.812+0000 7f11f52ed700 1 --2- 172.21.15.122:0/3310957697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f0101e60 0x7f11f01022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:53.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.812+0000 7f11f52ed700 1 -- 172.21.15.122:0/3310957697 >> 172.21.15.122:0/3310957697 conn(0x7f11f00776c0 msgr2=0x7f11f0077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:53.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.812+0000 7f11f52ed700 1 -- 172.21.15.122:0/3310957697 shutdown_connections 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.812+0000 7f11f52ed700 1 -- 172.21.15.122:0/3310957697 wait complete. 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11f52ed700 1 Processor -- start 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11f52ed700 1 -- start start 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11f52ed700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11f0101e60 0x7f11f019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11f52ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f0102820 0x7f11f019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11f52ed700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11f0108580 0x7f11f0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11f52ed700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11f0113670 con 0x7f11f0108580 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11f52ed700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f11f01134f0 con 0x7f11f0101e60 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11f52ed700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f11f01137f0 con 0x7f11f0102820 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11ef7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11f0108580 0x7f11f0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11ef7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11f0108580 0x7f11f0199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49762/0 (socket says 172.21.15.122:49762) 2024-05-03T17:54:53.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11ef7fe700 1 -- 172.21.15.122:0/2895981202 learned_addr learned my addr 172.21.15.122:0/2895981202 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.813+0000 7f11eeffd700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11f0101e60 0x7f11f019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11ee7fc700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f0102820 0x7f11f019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11ef7fe700 1 -- 172.21.15.122:0/2895981202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f0102820 msgr2=0x7f11f019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11ef7fe700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f0102820 0x7f11f019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11ef7fe700 1 -- 172.21.15.122:0/2895981202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11f0101e60 msgr2=0x7f11f019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11ef7fe700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11f0101e60 0x7f11f019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11ef7fe700 1 -- 172.21.15.122:0/2895981202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f019a010 con 0x7f11f0108580 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11ef7fe700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11f0108580 0x7f11f0199830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f11e800b830 tx=0x7f11e8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11d7fff700 1 -- 172.21.15.122:0/2895981202 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8021070 con 0x7f11f0108580 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11d7fff700 1 -- 172.21.15.122:0/2895981202 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e8007e30 con 0x7f11f0108580 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11f52ed700 1 -- 172.21.15.122:0/2895981202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f019a2a0 con 0x7f11f0108580 2024-05-03T17:54:53.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11d7fff700 1 -- 172.21.15.122:0/2895981202 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8004700 con 0x7f11f0108580 2024-05-03T17:54:53.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.814+0000 7f11f52ed700 1 -- 172.21.15.122:0/2895981202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f01a6510 con 0x7f11f0108580 2024-05-03T17:54:53.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.815+0000 7f11f52ed700 1 -- 172.21.15.122:0/2895981202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f006b690 con 0x7f11f0108580 2024-05-03T17:54:53.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.816+0000 7f11d7fff700 1 -- 172.21.15.122:0/2895981202 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f11e8003d70 con 0x7f11f0108580 2024-05-03T17:54:53.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.817+0000 7f11d7fff700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f11d80b2910 0x7f11d80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:53.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.817+0000 7f11eeffd700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f11d80b2910 0x7f11d80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:53.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.817+0000 7f11d7fff700 1 -- 172.21.15.122:0/2895981202 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f11e800f020 con 0x7f11f0108580 2024-05-03T17:54:53.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.818+0000 7f11eeffd700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f11d80b2910 0x7f11d80b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f11f0078c40 tx=0x7f11e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:53.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:53.821+0000 7f11d7fff700 1 -- 172.21.15.122:0/2895981202 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f11e802a020 con 0x7f11f0108580 2024-05-03T17:54:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.076+0000 7f11f52ed700 1 -- 172.21.15.122:0/2895981202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f11f002e510 con 0x7f11f0108580 2024-05-03T17:54:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.076+0000 7f11d7fff700 1 -- 172.21.15.122:0/2895981202 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f11e80ab8c0 con 0x7f11f0108580 2024-05-03T17:54:54.071 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:54:54.071 INFO:teuthology.orchestra.run.smithi122.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-03T17:54:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 -- 172.21.15.122:0/2895981202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f11d80b2910 msgr2=0x7f11d80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f11d80b2910 0x7f11d80b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f11f0078c40 tx=0x7f11e000b410 comp rx=0 tx=0).stop 2024-05-03T17:54:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 -- 172.21.15.122:0/2895981202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11f0108580 msgr2=0x7f11f0199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11f0108580 0x7f11f0199830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f11e800b830 tx=0x7f11e8004ce0 comp rx=0 tx=0).stop 2024-05-03T17:54:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 -- 172.21.15.122:0/2895981202 shutdown_connections 2024-05-03T17:54:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f11d80b2910 0x7f11d80b4dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11f0108580 0x7f11f0199830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11f0101e60 0x7f11f019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 --2- 172.21.15.122:0/2895981202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f0102820 0x7f11f019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 -- 172.21.15.122:0/2895981202 >> 172.21.15.122:0/2895981202 conn(0x7f11f00776c0 msgr2=0x7f11f0100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 -- 172.21.15.122:0/2895981202 shutdown_connections 2024-05-03T17:54:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:54.078+0000 7f11f52ed700 1 -- 172.21.15.122:0/2895981202 wait complete. 2024-05-03T17:54:54.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:54 smithi122 ceph-mon[29744]: pgmap v356: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:54.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:54 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2895981202' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T17:54:54.756 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-03T17:54:54.757 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:54:54.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:54 smithi149 ceph-mon[33409]: pgmap v356: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:54.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:54 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2895981202' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T17:54:54.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:54 smithi192 ceph-mon[33753]: pgmap v356: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:54:54.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:54 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2895981202' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T17:54:54.993 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:56.609 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:56 smithi122 ceph-mon[29744]: pgmap v357: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:56.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.642+0000 7f39dd3ab700 1 -- 172.21.15.122:0/3664204922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d8104180 msgr2=0x7f39d81103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:56.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.642+0000 7f39dd3ab700 1 --2- 172.21.15.122:0/3664204922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d8104180 0x7f39d81103f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f39d000b0a0 tx=0x7f39d000b3b0 comp rx=0 tx=0).stop 2024-05-03T17:54:56.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.642+0000 7f39dd3ab700 1 -- 172.21.15.122:0/3664204922 shutdown_connections 2024-05-03T17:54:56.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.642+0000 7f39dd3ab700 1 --2- 172.21.15.122:0/3664204922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d8104180 0x7f39d81103f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:56.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.642+0000 7f39dd3ab700 1 --2- 172.21.15.122:0/3664204922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d8109660 0x7f39d8109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:56.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.642+0000 7f39dd3ab700 1 --2- 172.21.15.122:0/3664204922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d81037c0 0x7f39d8103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:56.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.642+0000 7f39dd3ab700 1 -- 172.21.15.122:0/3664204922 >> 172.21.15.122:0/3664204922 conn(0x7f39d807b430 msgr2=0x7f39d807b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:56.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.642+0000 7f39dd3ab700 1 -- 172.21.15.122:0/3664204922 shutdown_connections 2024-05-03T17:54:56.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.642+0000 7f39dd3ab700 1 -- 172.21.15.122:0/3664204922 wait complete. 2024-05-03T17:54:56.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.643+0000 7f39dd3ab700 1 Processor -- start 2024-05-03T17:54:56.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.643+0000 7f39dd3ab700 1 -- start start 2024-05-03T17:54:56.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.643+0000 7f39dd3ab700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d81037c0 0x7f39d819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:56.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.643+0000 7f39dd3ab700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d8104180 0x7f39d819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:56.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.643+0000 7f39dd3ab700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d8109660 0x7f39d8199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:56.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.643+0000 7f39dd3ab700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8113740 con 0x7f39d8109660 2024-05-03T17:54:56.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.643+0000 7f39dd3ab700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81135c0 con 0x7f39d81037c0 2024-05-03T17:54:56.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.643+0000 7f39dd3ab700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81138c0 con 0x7f39d8104180 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d8109660 0x7f39d8199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d8109660 0x7f39d8199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49786/0 (socket says 172.21.15.122:49786) 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d77fe700 1 -- 172.21.15.122:0/456997745 learned_addr learned my addr 172.21.15.122:0/456997745 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d6ffd700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d81037c0 0x7f39d819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d67fc700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d8104180 0x7f39d819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d77fe700 1 -- 172.21.15.122:0/456997745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d8104180 msgr2=0x7f39d819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d77fe700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d8104180 0x7f39d819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d77fe700 1 -- 172.21.15.122:0/456997745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d81037c0 msgr2=0x7f39d819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d77fe700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d81037c0 0x7f39d819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d77fe700 1 -- 172.21.15.122:0/456997745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d819a180 con 0x7f39d8109660 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d6ffd700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d81037c0 0x7f39d819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39d77fe700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d8109660 0x7f39d8199a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f39d000b830 tx=0x7f39d0004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39bffff700 1 -- 172.21.15.122:0/456997745 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0021070 con 0x7f39d8109660 2024-05-03T17:54:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39bffff700 1 -- 172.21.15.122:0/456997745 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d0007e30 con 0x7f39d8109660 2024-05-03T17:54:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39dd3ab700 1 -- 172.21.15.122:0/456997745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d819a410 con 0x7f39d8109660 2024-05-03T17:54:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39bffff700 1 -- 172.21.15.122:0/456997745 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0004660 con 0x7f39d8109660 2024-05-03T17:54:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.644+0000 7f39dd3ab700 1 -- 172.21.15.122:0/456997745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d81a6730 con 0x7f39d8109660 2024-05-03T17:54:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.645+0000 7f39dd3ab700 1 -- 172.21.15.122:0/456997745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d806a190 con 0x7f39d8109660 2024-05-03T17:54:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.646+0000 7f39bffff700 1 -- 172.21.15.122:0/456997745 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f39d0003c00 con 0x7f39d8109660 2024-05-03T17:54:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.647+0000 7f39bffff700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f39c00b2990 0x7f39c00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.647+0000 7f39d6ffd700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f39c00b2990 0x7f39c00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.647+0000 7f39bffff700 1 -- 172.21.15.122:0/456997745 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f39d000f020 con 0x7f39d8109660 2024-05-03T17:54:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.648+0000 7f39d6ffd700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f39c00b2990 0x7f39c00b4e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f39cc005e10 tx=0x7f39cc005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.651+0000 7f39bffff700 1 -- 172.21.15.122:0/456997745 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f39d002a020 con 0x7f39d8109660 2024-05-03T17:54:56.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:56 smithi149 ceph-mon[33409]: pgmap v357: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:56.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:56 smithi192 ceph-mon[33753]: pgmap v357: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:54:56.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.904+0000 7f39dd3ab700 1 -- 172.21.15.122:0/456997745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39d802e510 con 0x7f39d8109660 2024-05-03T17:54:56.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.905+0000 7f39bffff700 1 -- 172.21.15.122:0/456997745 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11892 (secure 0 0 0) 0x7f39d00aba20 con 0x7f39d8109660 2024-05-03T17:54:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:54:56.900 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/555236113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":555236113},{"type":"v1","addr":"172.21.15.122:6839","nonce":555236113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1783292878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1783292878},{"type":"v1","addr":"172.21.15.122:6841","nonce":1783292878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T17:52:37.616728+0000","modified":"2024-05-03T17:52:56.867078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24463,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24410":{"gid":24410,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1529632960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1529632960},{"type":"v1","addr":"172.21.15.149:6839","nonce":1529632960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1429818440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1429818440},{"type":"v1","addr":"172.21.15.192:6839","nonce":1429818440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24457,24463]},"id":1}]} 2024-05-03T17:54:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.907+0000 7f39dd3ab700 1 -- 172.21.15.122:0/456997745 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f39c00b2990 msgr2=0x7f39c00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.907+0000 7f39dd3ab700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f39c00b2990 0x7f39c00b4e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f39cc005e10 tx=0x7f39cc005d60 comp rx=0 tx=0).stop 2024-05-03T17:54:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.907+0000 7f39dd3ab700 1 -- 172.21.15.122:0/456997745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d8109660 msgr2=0x7f39d8199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.907+0000 7f39dd3ab700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d8109660 0x7f39d8199a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f39d000b830 tx=0x7f39d0004d90 comp rx=0 tx=0).stop 2024-05-03T17:54:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.907+0000 7f39dd3ab700 1 -- 172.21.15.122:0/456997745 shutdown_connections 2024-05-03T17:54:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.907+0000 7f39dd3ab700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f39c00b2990 0x7f39c00b4e50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.907+0000 7f39dd3ab700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d8109660 0x7f39d8199a20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:56.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.907+0000 7f39dd3ab700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d81037c0 0x7f39d819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:56.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.907+0000 7f39dd3ab700 1 --2- 172.21.15.122:0/456997745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d8104180 0x7f39d819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:56.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.907+0000 7f39dd3ab700 1 -- 172.21.15.122:0/456997745 >> 172.21.15.122:0/456997745 conn(0x7f39d807b430 msgr2=0x7f39d8102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:56.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.908+0000 7f39dd3ab700 1 -- 172.21.15.122:0/456997745 shutdown_connections 2024-05-03T17:54:56.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:56.908+0000 7f39dd3ab700 1 -- 172.21.15.122:0/456997745 wait complete. 2024-05-03T17:54:56.904 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 24 2024-05-03T17:54:57.531 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:57 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/456997745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:54:57.559 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:54:57.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:57 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/456997745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:54:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:54:57.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:57 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/456997745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:54:58.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:54:58 smithi122 ceph-mon[29744]: pgmap v358: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:58.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:54:58 smithi149 ceph-mon[33409]: pgmap v358: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:58.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:54:58 smithi192 ceph-mon[33753]: pgmap v358: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.472+0000 7fe18a379700 1 -- 172.21.15.122:0/4255908552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe184102b00 msgr2=0x7fe184102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.472+0000 7fe18a379700 1 --2- 172.21.15.122:0/4255908552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe184102b00 0x7fe184102f80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe178009990 tx=0x7fe178009ca0 comp rx=0 tx=0).stop 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.472+0000 7fe18a379700 1 -- 172.21.15.122:0/4255908552 shutdown_connections 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.472+0000 7fe18a379700 1 --2- 172.21.15.122:0/4255908552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1841034c0 0x7fe18410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.472+0000 7fe18a379700 1 --2- 172.21.15.122:0/4255908552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe184108470 0x7fe184108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.472+0000 7fe18a379700 1 --2- 172.21.15.122:0/4255908552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe184102b00 0x7fe184102f80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.472+0000 7fe18a379700 1 -- 172.21.15.122:0/4255908552 >> 172.21.15.122:0/4255908552 conn(0x7fe1840789a0 msgr2=0x7fe1840ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.472+0000 7fe18a379700 1 -- 172.21.15.122:0/4255908552 shutdown_connections 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.472+0000 7fe18a379700 1 -- 172.21.15.122:0/4255908552 wait complete. 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.473+0000 7fe18a379700 1 Processor -- start 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.473+0000 7fe18a379700 1 -- start start 2024-05-03T17:54:59.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.473+0000 7fe18a379700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe184102b00 0x7fe184198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.473+0000 7fe18a379700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1841034c0 0x7fe184198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe18a379700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe184108470 0x7fe1841991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe18a379700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe184112560 con 0x7fe184102b00 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe18a379700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe1841123e0 con 0x7fe1841034c0 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe18a379700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1841126e0 con 0x7fe184108470 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe183fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe184102b00 0x7fe184198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe183fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe184102b00 0x7fe184198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40652/0 (socket says 172.21.15.122:40652) 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe183fff700 1 -- 172.21.15.122:0/866701932 learned_addr learned my addr 172.21.15.122:0/866701932 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe188916700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe184108470 0x7fe1841991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe1837fe700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1841034c0 0x7fe184198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe183fff700 1 -- 172.21.15.122:0/866701932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe184108470 msgr2=0x7fe1841991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe183fff700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe184108470 0x7fe1841991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe183fff700 1 -- 172.21.15.122:0/866701932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1841034c0 msgr2=0x7fe184198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:59.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe183fff700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1841034c0 0x7fe184198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:59.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe183fff700 1 -- 172.21.15.122:0/866701932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1841a7fe0 con 0x7fe184102b00 2024-05-03T17:54:59.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe1837fe700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1841034c0 0x7fe184198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:54:59.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe188916700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe184108470 0x7fe1841991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:54:59.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe183fff700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe184102b00 0x7fe184198770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe17400c9b0 tx=0x7fe17400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:59.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe1817fa700 1 -- 172.21.15.122:0/866701932 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174007b00 con 0x7fe184102b00 2024-05-03T17:54:59.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe18a379700 1 -- 172.21.15.122:0/866701932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1841a82d0 con 0x7fe184102b00 2024-05-03T17:54:59.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe1817fa700 1 -- 172.21.15.122:0/866701932 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe174007ca0 con 0x7fe184102b00 2024-05-03T17:54:59.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe1817fa700 1 -- 172.21.15.122:0/866701932 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174018730 con 0x7fe184102b00 2024-05-03T17:54:59.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.474+0000 7fe18a379700 1 -- 172.21.15.122:0/866701932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1841a8810 con 0x7fe184102b00 2024-05-03T17:54:59.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.475+0000 7fe18a379700 1 -- 172.21.15.122:0/866701932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18406a190 con 0x7fe184102b00 2024-05-03T17:54:59.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.477+0000 7fe1817fa700 1 -- 172.21.15.122:0/866701932 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe17400f450 con 0x7fe184102b00 2024-05-03T17:54:59.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.477+0000 7fe1817fa700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe16c0b2910 0x7fe16c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:54:59.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.477+0000 7fe1837fe700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe16c0b2910 0x7fe16c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:54:59.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.477+0000 7fe1817fa700 1 -- 172.21.15.122:0/866701932 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fe1740e2570 con 0x7fe184102b00 2024-05-03T17:54:59.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.478+0000 7fe1837fe700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe16c0b2910 0x7fe16c0b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe178000c00 tx=0x7fe178005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:54:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.481+0000 7fe1817fa700 1 -- 172.21.15.122:0/866701932 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe174014030 con 0x7fe184102b00 2024-05-03T17:54:59.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.733+0000 7fe18a379700 1 -- 172.21.15.122:0/866701932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe18402e510 con 0x7fe184102b00 2024-05-03T17:54:59.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.733+0000 7fe1817fa700 1 -- 172.21.15.122:0/866701932 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11892 (secure 0 0 0) 0x7fe1740aa2c0 con 0x7fe184102b00 2024-05-03T17:54:59.729 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:54:59.729 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/555236113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":555236113},{"type":"v1","addr":"172.21.15.122:6839","nonce":555236113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1783292878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1783292878},{"type":"v1","addr":"172.21.15.122:6841","nonce":1783292878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T17:52:37.616728+0000","modified":"2024-05-03T17:52:56.867078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24463,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24410":{"gid":24410,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1529632960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1529632960},{"type":"v1","addr":"172.21.15.149:6839","nonce":1529632960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1429818440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1429818440},{"type":"v1","addr":"172.21.15.192:6839","nonce":1429818440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24457,24463]},"id":1}]} 2024-05-03T17:54:59.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 -- 172.21.15.122:0/866701932 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe16c0b2910 msgr2=0x7fe16c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:59.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe16c0b2910 0x7fe16c0b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe178000c00 tx=0x7fe178005fb0 comp rx=0 tx=0).stop 2024-05-03T17:54:59.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 -- 172.21.15.122:0/866701932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe184102b00 msgr2=0x7fe184198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:54:59.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe184102b00 0x7fe184198770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe17400c9b0 tx=0x7fe17400cd70 comp rx=0 tx=0).stop 2024-05-03T17:54:59.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 -- 172.21.15.122:0/866701932 shutdown_connections 2024-05-03T17:54:59.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe16c0b2910 0x7fe16c0b4dd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:59.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe184108470 0x7fe1841991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:59.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe184102b00 0x7fe184198770 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:59.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 --2- 172.21.15.122:0/866701932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1841034c0 0x7fe184198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:54:59.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 -- 172.21.15.122:0/866701932 >> 172.21.15.122:0/866701932 conn(0x7fe1840789a0 msgr2=0x7fe184100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:54:59.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 -- 172.21.15.122:0/866701932 shutdown_connections 2024-05-03T17:54:59.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:54:59.736+0000 7fe18a379700 1 -- 172.21.15.122:0/866701932 wait complete. 2024-05-03T17:54:59.733 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 24 2024-05-03T17:55:00.387 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T17:55:00.624 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:00.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:00 smithi122 ceph-mon[29744]: pgmap v359: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:00 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/866701932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:00.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:00 smithi149 ceph-mon[33409]: pgmap v359: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:00.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:00 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/866701932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:00.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:00 smithi192 ceph-mon[33753]: pgmap v359: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:00.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:00 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/866701932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:02.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.325+0000 7f26a1621700 1 -- 172.21.15.122:0/886165698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f269c103080 msgr2=0x7f269c10f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:02.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.325+0000 7f26a1621700 1 --2- 172.21.15.122:0/886165698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f269c103080 0x7f269c10f1f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f269400b0a0 tx=0x7f269400b3b0 comp rx=0 tx=0).stop 2024-05-03T17:55:02.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.326+0000 7f26a1621700 1 -- 172.21.15.122:0/886165698 shutdown_connections 2024-05-03T17:55:02.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.326+0000 7f26a1621700 1 --2- 172.21.15.122:0/886165698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f269c103080 0x7f269c10f1f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:02.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.326+0000 7f26a1621700 1 --2- 172.21.15.122:0/886165698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c108510 0x7f269c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:02.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.326+0000 7f26a1621700 1 --2- 172.21.15.122:0/886165698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c1026c0 0x7f269c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:02.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.326+0000 7f26a1621700 1 -- 172.21.15.122:0/886165698 >> 172.21.15.122:0/886165698 conn(0x7f269c079100 msgr2=0x7f269c079510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:02.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.326+0000 7f26a1621700 1 -- 172.21.15.122:0/886165698 shutdown_connections 2024-05-03T17:55:02.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.326+0000 7f26a1621700 1 -- 172.21.15.122:0/886165698 wait complete. 2024-05-03T17:55:02.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.326+0000 7f26a1621700 1 Processor -- start 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.326+0000 7f26a1621700 1 -- start start 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f26a1621700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f269c1026c0 0x7f269c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f26a1621700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c103080 0x7f269c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f26a1621700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c108510 0x7f269c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f26a1621700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1125a0 con 0x7f269c1026c0 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f26a1621700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f269c112420 con 0x7f269c108510 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f26a1621700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f269c112720 con 0x7f269c103080 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f269c1026c0 0x7f269c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269a7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c103080 0x7f269c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f269c1026c0 0x7f269c19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40674/0 (socket says 172.21.15.122:40674) 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269affd700 1 -- 172.21.15.122:0/2672040606 learned_addr learned my addr 172.21.15.122:0/2672040606 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269b7fe700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c108510 0x7f269c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269affd700 1 -- 172.21.15.122:0/2672040606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c103080 msgr2=0x7f269c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269affd700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c103080 0x7f269c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269affd700 1 -- 172.21.15.122:0/2672040606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c108510 msgr2=0x7f269c199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269affd700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c108510 0x7f269c199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269affd700 1 -- 172.21.15.122:0/2672040606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c199ff0 con 0x7f269c1026c0 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269b7fe700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c108510 0x7f269c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f269affd700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f269c1026c0 0x7f269c19f6d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f268c00d980 tx=0x7f268c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f2683fff700 1 -- 172.21.15.122:0/2672040606 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c017070 con 0x7f269c1026c0 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f26a1621700 1 -- 172.21.15.122:0/2672040606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269c19a2e0 con 0x7f269c1026c0 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f2683fff700 1 -- 172.21.15.122:0/2672040606 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f268c005020 con 0x7f269c1026c0 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.327+0000 7f2683fff700 1 -- 172.21.15.122:0/2672040606 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c009b50 con 0x7f269c1026c0 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.328+0000 7f26a1621700 1 -- 172.21.15.122:0/2672040606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c1a65a0 con 0x7f269c1026c0 2024-05-03T17:55:02.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.329+0000 7f26a1621700 1 -- 172.21.15.122:0/2672040606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f269c06a190 con 0x7f269c1026c0 2024-05-03T17:55:02.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.330+0000 7f2683fff700 1 -- 172.21.15.122:0/2672040606 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f268c0051d0 con 0x7f269c1026c0 2024-05-03T17:55:02.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.330+0000 7f2683fff700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f26840b2910 0x7f26840b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:02.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.330+0000 7f269a7fc700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f26840b2910 0x7f26840b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:02.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.330+0000 7f2683fff700 1 -- 172.21.15.122:0/2672040606 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f268c0e21d0 con 0x7f269c1026c0 2024-05-03T17:55:02.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.331+0000 7f269a7fc700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f26840b2910 0x7f26840b4dd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f269c079670 tx=0x7f269000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:02.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.335+0000 7f2683fff700 1 -- 172.21.15.122:0/2672040606 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f268c020020 con 0x7f269c1026c0 2024-05-03T17:55:02.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.540+0000 7f26a1621700 1 -- 172.21.15.122:0/2672040606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f269c1a67d0 con 0x7f269c1026c0 2024-05-03T17:55:02.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.542+0000 7f2683fff700 1 -- 172.21.15.122:0/2672040606 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22622 (secure 0 0 0) 0x7f268c0a9e90 con 0x7f269c1026c0 2024-05-03T17:55:02.538 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:55:02.538 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":79,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","created":"2024-05-03T17:44:17.415571+0000","modified":"2024-05-03T17:53:19.863832+0000","last_up_change":"2024-05-03T17:51:28.839117+0000","last_in_change":"2024-05-03T17:51:07.954070+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T17:47:40.539967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T17:52:31.646919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T17:52:34.503067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4b08628-0a29-44aa-b8b9-ed9657f12be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6803","nonce":1329364610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6805","nonce":1329364610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6809","nonce":1329364610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6807","nonce":1329364610}]},"public_addr":"172.21.15.122:6803/1329364610","cluster_addr":"172.21.15.122:6805/1329364610","heartbeat_back_addr":"172.21.15.122:6809/1329364610","heartbeat_front_addr":"172.21.15.122:6807/1329364610","state":["exists","up"]},{"osd":1,"uuid":"ba550787-ce76-4944-8ebd-97e19590bb3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6801","nonce":1402463302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6803","nonce":1402463302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6807","nonce":1402463302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6805","nonce":1402463302}]},"public_addr":"172.21.15.149:6801/1402463302","cluster_addr":"172.21.15.149:6803/1402463302","heartbeat_back_addr":"172.21.15.149:6807/1402463302","heartbeat_front_addr":"172.21.15.149:6805/1402463302","state":["exists","up"]},{"osd":2,"uuid":"35ae7c47-d710-47c5-b0ec-91fa12b5be07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6801","nonce":3757191110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6803","nonce":3757191110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6807","nonce":3757191110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6805","nonce":3757191110}]},"public_addr":"172.21.15.192:6801/3757191110","cluster_addr":"172.21.15.192:6803/3757191110","heartbeat_back_addr":"172.21.15.192:6807/3757191110","heartbeat_front_addr":"172.21.15.192:6805/3757191110","state":["exists","up"]},{"osd":3,"uuid":"9973c0c1-ae73-497a-845e-664a20666df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6811","nonce":2359143764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6813","nonce":2359143764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6817","nonce":2359143764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6815","nonce":2359143764}]},"public_addr":"172.21.15.122:6811/2359143764","cluster_addr":"172.21.15.122:6813/2359143764","heartbeat_back_addr":"172.21.15.122:6817/2359143764","heartbeat_front_addr":"172.21.15.122:6815/2359143764","state":["exists","up"]},{"osd":4,"uuid":"da71dc18-94b9-4f7d-a061-7cc7b77db282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3627037532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3627037532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3627037532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3627037532}]},"public_addr":"172.21.15.149:6809/3627037532","cluster_addr":"172.21.15.149:6811/3627037532","heartbeat_back_addr":"172.21.15.149:6815/3627037532","heartbeat_front_addr":"172.21.15.149:6813/3627037532","state":["exists","up"]},{"osd":5,"uuid":"bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6809","nonce":455779485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6811","nonce":455779485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6815","nonce":455779485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6813","nonce":455779485}]},"public_addr":"172.21.15.192:6809/455779485","cluster_addr":"172.21.15.192:6811/455779485","heartbeat_back_addr":"172.21.15.192:6815/455779485","heartbeat_front_addr":"172.21.15.192:6813/455779485","state":["exists","up"]},{"osd":6,"uuid":"4150bfe5-a0e4-4bae-914a-75598f017519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6819","nonce":3419421998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6821","nonce":3419421998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6825","nonce":3419421998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6823","nonce":3419421998}]},"public_addr":"172.21.15.122:6819/3419421998","cluster_addr":"172.21.15.122:6821/3419421998","heartbeat_back_addr":"172.21.15.122:6825/3419421998","heartbeat_front_addr":"172.21.15.122:6823/3419421998","state":["exists","up"]},{"osd":7,"uuid":"71e01812-e511-44a0-a143-2afca510ae8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6817","nonce":139747549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6819","nonce":139747549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6823","nonce":139747549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6821","nonce":139747549}]},"public_addr":"172.21.15.149:6817/139747549","cluster_addr":"172.21.15.149:6819/139747549","heartbeat_back_addr":"172.21.15.149:6823/139747549","heartbeat_front_addr":"172.21.15.149:6821/139747549","state":["exists","up"]},{"osd":8,"uuid":"5294516d-fac6-41d0-a635-6bc1d7eca585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6817","nonce":2080816744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6819","nonce":2080816744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6823","nonce":2080816744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6821","nonce":2080816744}]},"public_addr":"172.21.15.192:6817/2080816744","cluster_addr":"172.21.15.192:6819/2080816744","heartbeat_back_addr":"172.21.15.192:6823/2080816744","heartbeat_front_addr":"172.21.15.192:6821/2080816744","state":["exists","up"]},{"osd":9,"uuid":"f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6827","nonce":1773691421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6829","nonce":1773691421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6833","nonce":1773691421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6831","nonce":1773691421}]},"public_addr":"172.21.15.122:6827/1773691421","cluster_addr":"172.21.15.122:6829/1773691421","heartbeat_back_addr":"172.21.15.122:6833/1773691421","heartbeat_front_addr":"172.21.15.122:6831/1773691421","state":["exists","up"]},{"osd":10,"uuid":"af88e50f-98f1-460d-856b-a920dd2ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6825","nonce":1954186137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6827","nonce":1954186137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6831","nonce":1954186137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6829","nonce":1954186137}]},"public_addr":"172.21.15.149:6825/1954186137","cluster_addr":"172.21.15.149:6827/1954186137","heartbeat_back_addr":"172.21.15.149:6831/1954186137","heartbeat_front_addr":"172.21.15.149:6829/1954186137","state":["exists","up"]},{"osd":11,"uuid":"b54c1b4d-cdb5-42d9-979a-47f9a870b5cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6825","nonce":2098474209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6827","nonce":2098474209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6831","nonce":2098474209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6829","nonce":2098474209}]},"public_addr":"172.21.15.192:6825/2098474209","cluster_addr":"172.21.15.192:6827/2098474209","heartbeat_back_addr":"172.21.15.192:6831/2098474209","heartbeat_front_addr":"172.21.15.192:6829/2098474209","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:46:50.049769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:13.518485+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:37.473023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:01.809640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:26.648514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:51.268056+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:16.294834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:42.079065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:07.609271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:33.885127+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:59.982708+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:51:26.461369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/2089907395":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6801/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:0/2001860625":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/1888284988":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/256574472":"2024-05-04T17:44:52.244049+0000","172.21.15.122:6801/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/64546794":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/584846864":"2024-05-04T17:45:18.454406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T17:55:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 -- 172.21.15.122:0/2672040606 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f26840b2910 msgr2=0x7f26840b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f26840b2910 0x7f26840b4dd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f269c079670 tx=0x7f269000a300 comp rx=0 tx=0).stop 2024-05-03T17:55:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 -- 172.21.15.122:0/2672040606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f269c1026c0 msgr2=0x7f269c19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f269c1026c0 0x7f269c19f6d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f268c00d980 tx=0x7f268c00dc90 comp rx=0 tx=0).stop 2024-05-03T17:55:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 -- 172.21.15.122:0/2672040606 shutdown_connections 2024-05-03T17:55:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f26840b2910 0x7f26840b4dd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c108510 0x7f269c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f269c1026c0 0x7f269c19f6d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 --2- 172.21.15.122:0/2672040606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c103080 0x7f269c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:02.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 -- 172.21.15.122:0/2672040606 >> 172.21.15.122:0/2672040606 conn(0x7f269c079100 msgr2=0x7f269c101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:02.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 -- 172.21.15.122:0/2672040606 shutdown_connections 2024-05-03T17:55:02.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:02.544+0000 7f26a1621700 1 -- 172.21.15.122:0/2672040606 wait complete. 2024-05-03T17:55:02.667 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:02 smithi192 ceph-mon[33753]: pgmap v360: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:02.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:02 smithi122 ceph-mon[29744]: pgmap v360: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:02.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:02 smithi149 ceph-mon[33409]: pgmap v360: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:03.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:55:03.408 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:03.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:03 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2672040606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:55:03.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:03 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2672040606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:55:03.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:03 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2672040606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:55:04.560 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:04 smithi122 ceph-mon[29744]: pgmap v361: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:04.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:04 smithi149 ceph-mon[33409]: pgmap v361: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:04.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:04 smithi192 ceph-mon[33753]: pgmap v361: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:05.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.069+0000 7fb415d8a700 1 -- 172.21.15.122:0/4266615719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb41006b180 msgr2=0x7fb41006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:05.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.069+0000 7fb415d8a700 1 --2- 172.21.15.122:0/4266615719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb41006b180 0x7fb41006b600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb404009990 tx=0x7fb404009ca0 comp rx=0 tx=0).stop 2024-05-03T17:55:05.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.069+0000 7fb415d8a700 1 -- 172.21.15.122:0/4266615719 shutdown_connections 2024-05-03T17:55:05.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.069+0000 7fb415d8a700 1 --2- 172.21.15.122:0/4266615719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41006bb40 0x7fb410110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:05.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.069+0000 7fb415d8a700 1 --2- 172.21.15.122:0/4266615719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb410109510 0x7fb4101098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:05.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.069+0000 7fb415d8a700 1 --2- 172.21.15.122:0/4266615719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb41006b180 0x7fb41006b600 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:05.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.069+0000 7fb415d8a700 1 -- 172.21.15.122:0/4266615719 >> 172.21.15.122:0/4266615719 conn(0x7fb410079350 msgr2=0x7fb410079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:05.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.069+0000 7fb415d8a700 1 -- 172.21.15.122:0/4266615719 shutdown_connections 2024-05-03T17:55:05.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.069+0000 7fb415d8a700 1 -- 172.21.15.122:0/4266615719 wait complete. 2024-05-03T17:55:05.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.070+0000 7fb415d8a700 1 Processor -- start 2024-05-03T17:55:05.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.070+0000 7fb415d8a700 1 -- start start 2024-05-03T17:55:05.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb415d8a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb41006b180 0x7fb41019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:05.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb415d8a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41006bb40 0x7fb41019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:05.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb415d8a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb410109510 0x7fb410199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:05.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb415d8a700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb410113720 con 0x7fb410109510 2024-05-03T17:55:05.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb415d8a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101135a0 con 0x7fb41006bb40 2024-05-03T17:55:05.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb415d8a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101138a0 con 0x7fb41006b180 2024-05-03T17:55:05.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb410109510 0x7fb410199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb410109510 0x7fb410199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40688/0 (socket says 172.21.15.122:40688) 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40ffff700 1 -- 172.21.15.122:0/2389428538 learned_addr learned my addr 172.21.15.122:0/2389428538 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40f7fe700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb41006b180 0x7fb41019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40effd700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41006bb40 0x7fb41019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40ffff700 1 -- 172.21.15.122:0/2389428538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb41006b180 msgr2=0x7fb41019f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40ffff700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb41006b180 0x7fb41019f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40ffff700 1 -- 172.21.15.122:0/2389428538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41006bb40 msgr2=0x7fb41019fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40ffff700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41006bb40 0x7fb41019fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40ffff700 1 -- 172.21.15.122:0/2389428538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41019a240 con 0x7fb410109510 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40ffff700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb410109510 0x7fb410199ae0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b970 tx=0x7fb40800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40cff9700 1 -- 172.21.15.122:0/2389428538 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408019070 con 0x7fb410109510 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb415d8a700 1 -- 172.21.15.122:0/2389428538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41019a530 con 0x7fb410109510 2024-05-03T17:55:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40cff9700 1 -- 172.21.15.122:0/2389428538 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb408014440 con 0x7fb410109510 2024-05-03T17:55:05.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb40cff9700 1 -- 172.21.15.122:0/2389428538 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40800ce60 con 0x7fb410109510 2024-05-03T17:55:05.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb415d8a700 1 -- 172.21.15.122:0/2389428538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101a67f0 con 0x7fb410109510 2024-05-03T17:55:05.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.072+0000 7fb415d8a700 1 -- 172.21.15.122:0/2389428538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41010b9f0 con 0x7fb410109510 2024-05-03T17:55:05.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.075+0000 7fb40cff9700 1 -- 172.21.15.122:0/2389428538 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb40801c070 con 0x7fb410109510 2024-05-03T17:55:05.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.075+0000 7fb40cff9700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb3f80b2910 0x7fb3f80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:05.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.076+0000 7fb40f7fe700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb3f80b2910 0x7fb3f80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:05.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.076+0000 7fb40cff9700 1 -- 172.21.15.122:0/2389428538 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fb4080e1af0 con 0x7fb410109510 2024-05-03T17:55:05.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.076+0000 7fb40f7fe700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb3f80b2910 0x7fb3f80b4dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb400006fd0 tx=0x7fb400008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.079+0000 7fb40cff9700 1 -- 172.21.15.122:0/2389428538 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb4080a98c0 con 0x7fb410109510 2024-05-03T17:55:05.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.326+0000 7fb415d8a700 1 -- 172.21.15.122:0/2389428538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb41019afd0 con 0x7fb410109510 2024-05-03T17:55:05.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.327+0000 7fb40cff9700 1 -- 172.21.15.122:0/2389428538 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11892 (secure 0 0 0) 0x7fb4080a91c0 con 0x7fb410109510 2024-05-03T17:55:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:55:05.322 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/555236113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":555236113},{"type":"v1","addr":"172.21.15.122:6839","nonce":555236113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1783292878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1783292878},{"type":"v1","addr":"172.21.15.122:6841","nonce":1783292878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T17:52:37.616728+0000","modified":"2024-05-03T17:52:56.867078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24463,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24410":{"gid":24410,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1529632960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1529632960},{"type":"v1","addr":"172.21.15.149:6839","nonce":1529632960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1429818440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1429818440},{"type":"v1","addr":"172.21.15.192:6839","nonce":1429818440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24457,24463]},"id":1}]} 2024-05-03T17:55:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 -- 172.21.15.122:0/2389428538 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb3f80b2910 msgr2=0x7fb3f80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb3f80b2910 0x7fb3f80b4dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb400006fd0 tx=0x7fb400008040 comp rx=0 tx=0).stop 2024-05-03T17:55:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 -- 172.21.15.122:0/2389428538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb410109510 msgr2=0x7fb410199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb410109510 0x7fb410199ae0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b970 tx=0x7fb40800bd30 comp rx=0 tx=0).stop 2024-05-03T17:55:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 -- 172.21.15.122:0/2389428538 shutdown_connections 2024-05-03T17:55:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb3f80b2910 0x7fb3f80b4dd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb410109510 0x7fb410199ae0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb41006b180 0x7fb41019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 --2- 172.21.15.122:0/2389428538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41006bb40 0x7fb41019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 -- 172.21.15.122:0/2389428538 >> 172.21.15.122:0/2389428538 conn(0x7fb410079350 msgr2=0x7fb410101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 -- 172.21.15.122:0/2389428538 shutdown_connections 2024-05-03T17:55:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:05.329+0000 7fb415d8a700 1 -- 172.21.15.122:0/2389428538 wait complete. 2024-05-03T17:55:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 24 2024-05-03T17:55:05.635 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:05 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2389428538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:05.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:05 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2389428538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:05.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:05 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2389428538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:05.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-03T17:52:37.616728+0000', 'modified': '2024-05-03T17:52:56.867078+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24481, 'mds_1': 24463, 'mds_2': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24410': {'gid': 24410, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/2599524712', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 2599524712}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 2599524712}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6833/899389349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6832', 'nonce': 899389349}, {'type': 'v1', 'addr': '172.21.15.192:6833', 'nonce': 899389349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6839/1529632960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 1529632960}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 1529632960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.192:6835/2756708496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 2756708496}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 2756708496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.192:6837/3476407415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 3476407415}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 3476407415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.192:6839/1429818440', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 1429818440}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 1429818440}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24481, 'qdb_cluster': [24481, 24457, 24463]} 2024-05-03T17:55:05.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-03T17:55:05.973 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-03T17:55:06.212 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:06.616 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:06 smithi122 ceph-mon[29744]: pgmap v362: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:06.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:06 smithi149 ceph-mon[33409]: pgmap v362: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:06.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:06 smithi192 ceph-mon[33753]: pgmap v362: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:07.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.808+0000 7f08b8430700 1 -- 172.21.15.122:0/2846951875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08b0102d60 msgr2=0x7f08b010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:07.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.808+0000 7f08b8430700 1 --2- 172.21.15.122:0/2846951875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08b0102d60 0x7f08b010f250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009990 tx=0x7f08a8009ca0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.808+0000 7f08b8430700 1 -- 172.21.15.122:0/2846951875 shutdown_connections 2024-05-03T17:55:07.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.808+0000 7f08b8430700 1 --2- 172.21.15.122:0/2846951875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08b0102d60 0x7f08b010f250 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.808+0000 7f08b8430700 1 --2- 172.21.15.122:0/2846951875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b0108410 0x7f08b01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.808+0000 7f08b8430700 1 --2- 172.21.15.122:0/2846951875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08b01023a0 0x7f08b0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.808+0000 7f08b8430700 1 -- 172.21.15.122:0/2846951875 >> 172.21.15.122:0/2846951875 conn(0x7f08b0079350 msgr2=0x7f08b0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:07.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.808+0000 7f08b8430700 1 -- 172.21.15.122:0/2846951875 shutdown_connections 2024-05-03T17:55:07.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.808+0000 7f08b8430700 1 -- 172.21.15.122:0/2846951875 wait complete. 2024-05-03T17:55:07.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.808+0000 7f08b8430700 1 Processor -- start 2024-05-03T17:55:07.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b8430700 1 -- start start 2024-05-03T17:55:07.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b8430700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08b01023a0 0x7f08b019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b8430700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08b0102d60 0x7f08b019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b8430700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b0108410 0x7f08b01a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b8430700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0112530 con 0x7f08b01023a0 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b8430700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01123b0 con 0x7f08b0102d60 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b8430700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01126b0 con 0x7f08b0108410 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b61cc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08b01023a0 0x7f08b019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b61cc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08b01023a0 0x7f08b019a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40706/0 (socket says 172.21.15.122:40706) 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b61cc700 1 -- 172.21.15.122:0/3111809312 learned_addr learned my addr 172.21.15.122:0/3111809312 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b69cd700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b0108410 0x7f08b01a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b59cb700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08b0102d60 0x7f08b019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b61cc700 1 -- 172.21.15.122:0/3111809312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b0108410 msgr2=0x7f08b01a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b61cc700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b0108410 0x7f08b01a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b61cc700 1 -- 172.21.15.122:0/3111809312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08b0102d60 msgr2=0x7f08b019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:07.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b61cc700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08b0102d60 0x7f08b019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b61cc700 1 -- 172.21.15.122:0/3111809312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b01a5ef0 con 0x7f08b01023a0 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b59cb700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08b0102d60 0x7f08b019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b69cd700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b0108410 0x7f08b01a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b61cc700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08b01023a0 0x7f08b019a930 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0060b0 tx=0x7f08ac007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08a77fe700 1 -- 172.21.15.122:0/3111809312 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac01a070 con 0x7f08b01023a0 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08b8430700 1 -- 172.21.15.122:0/3111809312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b01a6120 con 0x7f08b01023a0 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08a77fe700 1 -- 172.21.15.122:0/3111809312 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08ac00de20 con 0x7f08b01023a0 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.809+0000 7f08a77fe700 1 -- 172.21.15.122:0/3111809312 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac014720 con 0x7f08b01023a0 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.810+0000 7f08b8430700 1 -- 172.21.15.122:0/3111809312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b01a6680 con 0x7f08b01023a0 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.812+0000 7f08a77fe700 1 -- 172.21.15.122:0/3111809312 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f08ac01d070 con 0x7f08b01023a0 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.812+0000 7f08a77fe700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f089c0b2910 0x7f089c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.812+0000 7f08b59cb700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f089c0b2910 0x7f089c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.812+0000 7f08a77fe700 1 -- 172.21.15.122:0/3111809312 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f08ac0e1b40 con 0x7f08b01023a0 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08b59cb700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f089c0b2910 0x7f089c0b4dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f08a0005fd0 tx=0x7f08a0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08c2cc3b80 1 Processor -- start 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08c2cc3b80 1 -- start start 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d56632910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632e50 0x562d56636890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56636dd0 0x562d56637210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562d565808f0 con 0x562d56632530 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562d564e5d50 con 0x562d56632e50 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562d5657c8b0 con 0x562d56636dd0 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08b61cc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d56632910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08b61cc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d56632910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40712/0 (socket says 172.21.15.122:40712) 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.813+0000 7f08b61cc700 1 -- 172.21.15.122:0/5775069 learned_addr learned my addr 172.21.15.122:0/5775069 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:07.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.814+0000 7f08b59cb700 1 --2- 172.21.15.122:0/5775069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632e50 0x562d56636890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.814+0000 7f08b61cc700 1 -- 172.21.15.122:0/5775069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56636dd0 msgr2=0x562d56637210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.814+0000 7f08b61cc700 1 --2- 172.21.15.122:0/5775069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56636dd0 0x562d56637210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.814+0000 7f08b61cc700 1 -- 172.21.15.122:0/5775069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632e50 msgr2=0x562d56636890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.814+0000 7f08b61cc700 1 --2- 172.21.15.122:0/5775069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632e50 0x562d56636890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.814+0000 7f08b61cc700 1 -- 172.21.15.122:0/5775069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d564cd880 con 0x562d56632530 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.814+0000 7f08b61cc700 1 --2- 172.21.15.122:0/5775069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d56632910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0df590 tx=0x7f08ac0a8c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4cd24427cee98b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.814+0000 7f08a4d70700 1 -- 172.21.15.122:0/5775069 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0ad4b0 con 0x562d56632530 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.814+0000 7f08a4d70700 1 -- 172.21.15.122:0/5775069 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08ac0b1f60 con 0x562d56632530 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.814+0000 7f08a4d70700 1 -- 172.21.15.122:0/5775069 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0b1d30 con 0x562d56632530 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.815+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/5775069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 msgr2=0x562d56632910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.815+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/5775069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d56632910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0df590 tx=0x7f08ac0a8c10 comp rx=0 tx=0).stop 2024-05-03T17:55:07.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.815+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/5775069 shutdown_connections 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.815+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/5775069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56636dd0 0x562d56637210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.815+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/5775069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d56632910 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.815+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/5775069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632e50 0x562d56636890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.815+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/5775069 >> 172.21.15.122:0/5775069 conn(0x562d5655ab20 msgr2=0x562d566289a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.815+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/5775069 shutdown_connections 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.815+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/5775069 wait complete. 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.815+0000 7f08c2cc3b80 1 -- start start 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.815+0000 7f08c2cc3b80 1 Processor -- start 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d5663e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d5663b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56636dd0 0x562d5663bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562d5663ef40 con 0x562d56632530 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562d56539e80 con 0x562d56636dd0 2024-05-03T17:55:07.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562d5663f0c0 con 0x562d56632e50 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08c2cc3b80 10 client.0 _ll_register_callbacks cb 0x562d56514560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08b61cc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d5663e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08b61cc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d5663e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40724/0 (socket says 172.21.15.122:40724) 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08b61cc700 1 -- 172.21.15.122:0/3202886358 learned_addr learned my addr 172.21.15.122:0/3202886358 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08b59cb700 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d5663b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08b61cc700 1 -- 172.21.15.122:0/3202886358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 msgr2=0x562d5663b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08b61cc700 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d5663b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08b61cc700 1 -- 172.21.15.122:0/3202886358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56636dd0 msgr2=0x562d5663bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08b61cc700 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56636dd0 0x562d5663bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08b61cc700 1 -- 172.21.15.122:0/3202886358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d56537710 con 0x562d56632530 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08b59cb700 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d5663b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08b61cc700 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d5663e1f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009990 tx=0x7f08ac0a9590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a4d4615a5716ca server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f0899ffb700 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-05-03T17:55:07.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f0899ffb700 1 -- 172.21.15.122:0/3202886358 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0c8060 con 0x562d56632530 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f0899ffb700 1 -- 172.21.15.122:0/3202886358 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f08ac0a8360 con 0x562d56632530 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/3202886358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ac0b1f60 con 0x562d56632530 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f0899ffb700 1 -- 172.21.15.122:0/3202886358 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f08ac088050 con 0x562d56632530 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08c2cc3b80 10 client.15012 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/3202886358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562d56649790 con 0x562d56632530 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.816+0000 7f0899ffb700 1 -- 172.21.15.122:0/3202886358 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08ac0b1f60 con 0x562d56632530 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.817+0000 7f0899ffb700 1 -- 172.21.15.122:0/3202886358 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f08ac0a7dd0 con 0x562d56632530 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.817+0000 7f08c2cc3b80 10 client.15012 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.817+0000 7f08c2cc3b80 10 client.15012 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.817+0000 7f08c2cc3b80 20 client.15012 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.817+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x562d5664b990 0x562d5664ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.817+0000 7f08c2cc3b80 4 client.15012 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T17:55:07.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.817+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/3202886358 --> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d5658e2d0 con 0x562d5664b990 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.817+0000 7f08b59cb700 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x562d5664b990 0x562d5664ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.818+0000 7f08b59cb700 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x562d5664b990 0x562d5664ce00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=484c50c3f365fc1 server_cookie=c851b3d836a8e267 in_seq=0 out_seq=0 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.818+0000 7f0899ffb700 10 client.15012 ms_handle_connect on v2:172.21.15.192:6834/2756708496 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.818+0000 7f0899ffb700 1 -- 172.21.15.122:0/3202886358 <== mds.2 v2:172.21.15.192:6834/2756708496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562d5658e2d0 con 0x562d5664b990 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.818+0000 7f0899ffb700 10 client.15012 handle_command_reply: tid=0 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.818+0000 7f08c2cc3b80 1 client.15012 shutdown 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.818+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/3202886358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 msgr2=0x562d5663e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.818+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d5663e1f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009990 tx=0x7f08ac0a9590 comp rx=0 tx=0).stop 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.819+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/3202886358 shutdown_connections 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.819+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x562d5664b990 0x562d5664ce00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.819+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56636dd0 0x562d5663bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.819+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56632530 0x562d5663e1f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.819+0000 7f0899ffb700 0 client.15012 ms_handle_reset on v2:172.21.15.192:6834/2756708496 2024-05-03T17:55:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.819+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/3202886358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d5663b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.819+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/3202886358 >> 172.21.15.122:0/3202886358 conn(0x562d5655ab20 msgr2=0x562d56638110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:07.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.819+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/3202886358 shutdown_connections 2024-05-03T17:55:07.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.819+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/3202886358 wait complete. 2024-05-03T17:55:07.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.819+0000 7f08c2cc3b80 20 client.15012 trim_cache size 0 max 16384 2024-05-03T17:55:07.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08c2cc3b80 1 Processor -- start 2024-05-03T17:55:07.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08c2cc3b80 1 -- start start 2024-05-03T17:55:07.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632530 0x562d5663a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d566371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d5664aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562d5664b140 con 0x562d56636dd0 2024-05-03T17:55:07.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562d5664b810 con 0x562d56632530 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562d5664bee0 con 0x562d56632e50 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08b69cd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d5664aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08b69cd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d5664aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40736/0 (socket says 172.21.15.122:40736) 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08b69cd700 1 -- 172.21.15.122:0/480715456 learned_addr learned my addr 172.21.15.122:0/480715456 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.839+0000 7f08b61cc700 1 --2- 172.21.15.122:0/480715456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632530 0x562d5663a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.840+0000 7f08b59cb700 1 --2- 172.21.15.122:0/480715456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d566371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.840+0000 7f08b69cd700 1 -- 172.21.15.122:0/480715456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 msgr2=0x562d566371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.840+0000 7f08b69cd700 1 --2- 172.21.15.122:0/480715456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d566371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.840+0000 7f08b69cd700 1 -- 172.21.15.122:0/480715456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632530 msgr2=0x562d5663a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.840+0000 7f08b69cd700 1 --2- 172.21.15.122:0/480715456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632530 0x562d5663a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.840+0000 7f08b69cd700 1 -- 172.21.15.122:0/480715456 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d56639b50 con 0x562d56636dd0 2024-05-03T17:55:07.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.840+0000 7f08b69cd700 1 --2- 172.21.15.122:0/480715456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d5664aae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f08a8003810 tx=0x7f08a8003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17f48b7f103f072 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:07.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.840+0000 7f089bfff700 1 -- 172.21.15.122:0/480715456 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8027070 con 0x562d56636dd0 2024-05-03T17:55:07.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.840+0000 7f089bfff700 1 -- 172.21.15.122:0/480715456 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08a8003da0 con 0x562d56636dd0 2024-05-03T17:55:07.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.840+0000 7f089bfff700 1 -- 172.21.15.122:0/480715456 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a800ba30 con 0x562d56636dd0 2024-05-03T17:55:07.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.841+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/480715456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 msgr2=0x562d5664aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:07.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.841+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/480715456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d5664aae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f08a8003810 tx=0x7f08a8003840 comp rx=0 tx=0).stop 2024-05-03T17:55:07.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.841+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/480715456 shutdown_connections 2024-05-03T17:55:07.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.841+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/480715456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d5664aae0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.841+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/480715456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632530 0x562d5663a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.841+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/480715456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d566371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.841+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/480715456 >> 172.21.15.122:0/480715456 conn(0x562d5655ab20 msgr2=0x562d5662cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:07.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.841+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/480715456 shutdown_connections 2024-05-03T17:55:07.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.841+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/480715456 wait complete. 2024-05-03T17:55:07.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.841+0000 7f08c2cc3b80 1 -- start start 2024-05-03T17:55:07.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.841+0000 7f08c2cc3b80 1 Processor -- start 2024-05-03T17:55:07.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632530 0x562d56656280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d566567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08c2cc3b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d56651230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562d5664cc80 con 0x562d56636dd0 2024-05-03T17:55:07.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562d5664c5b0 con 0x562d56632530 2024-05-03T17:55:07.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08c2cc3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562d5664d350 con 0x562d56632e50 2024-05-03T17:55:07.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08c2cc3b80 10 client.0 _ll_register_callbacks cb 0x562d56514560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:07.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b69cd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d56651230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b69cd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d56651230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40748/0 (socket says 172.21.15.122:40748) 2024-05-03T17:55:07.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b69cd700 1 -- 172.21.15.122:0/2560107887 learned_addr learned my addr 172.21.15.122:0/2560107887 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:07.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b59cb700 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d566567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b61cc700 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632530 0x562d56656280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b69cd700 1 -- 172.21.15.122:0/2560107887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 msgr2=0x562d566567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:07.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b69cd700 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d566567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b69cd700 1 -- 172.21.15.122:0/2560107887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632530 msgr2=0x562d56656280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:07.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b69cd700 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632530 0x562d56656280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b69cd700 1 -- 172.21.15.122:0/2560107887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d56651770 con 0x562d56636dd0 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b59cb700 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d566567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08b69cd700 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d56651230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009350 tx=0x7f08a8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa6441d88ed95ac server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f0899ffb700 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f0899ffb700 1 -- 172.21.15.122:0/2560107887 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a802f060 con 0x562d56636dd0 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/2560107887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a8003da0 con 0x562d56636dd0 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.842+0000 7f0899ffb700 1 -- 172.21.15.122:0/2560107887 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f08a800b7d0 con 0x562d56636dd0 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f0899ffb700 1 -- 172.21.15.122:0/2560107887 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f08a8027070 con 0x562d56636dd0 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f08c2cc3b80 10 client.15018 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/2560107887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562d566b75c0 con 0x562d56636dd0 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f0899ffb700 1 -- 172.21.15.122:0/2560107887 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08a8003da0 con 0x562d56636dd0 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f0899ffb700 1 -- 172.21.15.122:0/2560107887 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f08a8034070 con 0x562d56636dd0 2024-05-03T17:55:07.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f08c2cc3b80 10 client.15018 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:07.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f08c2cc3b80 10 client.15018 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T17:55:07.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f08c2cc3b80 20 client.15018 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:07.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x562d566bbbe0 0x562d566ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:07.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f08c2cc3b80 4 client.15018 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T17:55:07.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/2560107887 --> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] -- command(tid 0: {"prefix": "status"}) -- 0x562d56588370 con 0x562d566bbbe0 2024-05-03T17:55:07.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.843+0000 7f08b61cc700 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x562d566bbbe0 0x562d566ba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:07.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08b61cc700 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x562d566bbbe0 0x562d566ba8f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bc8bd0240c15590 server_cookie=bc71ac86a427c283 in_seq=0 out_seq=0 2024-05-03T17:55:07.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f0899ffb700 10 client.15018 ms_handle_connect on v2:172.21.15.192:6834/2756708496 2024-05-03T17:55:07.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f0899ffb700 1 -- 172.21.15.122:0/2560107887 <== mds.2 v2:172.21.15.192:6834/2756708496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562d56588370 con 0x562d566bbbe0 2024-05-03T17:55:07.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f0899ffb700 10 client.15018 handle_command_reply: tid=0 2024-05-03T17:55:07.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08c2cc3b80 1 client.15018 shutdown 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/2560107887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 msgr2=0x562d56651230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d56651230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009350 tx=0x7f08a8009ca0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/2560107887 shutdown_connections 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x562d566bbbe0 0x562d566ba8f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d56636dd0 0x562d56651230 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d56632530 0x562d56656280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08c2cc3b80 1 --2- 172.21.15.122:0/2560107887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d56632e50 0x562d566567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/2560107887 >> 172.21.15.122:0/2560107887 conn(0x562d5655ab20 msgr2=0x562d5662cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f0899ffb700 0 client.15018 ms_handle_reset on v2:172.21.15.192:6834/2756708496 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/2560107887 shutdown_connections 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.844+0000 7f08c2cc3b80 1 -- 172.21.15.122:0/2560107887 wait complete. 2024-05-03T17:55:07.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08c2cc3b80 20 client.15018 trim_cache size 0 max 16384 2024-05-03T17:55:07.847 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-05-03T17:55:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "a46fc846-0974-11ef-bc97-c7b262605968", 2024-05-03T17:55:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 2, 2024-05-03T17:55:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24457, 2024-05-03T17:55:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-05-03T17:55:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-05-03T17:55:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-05-03T17:55:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 137.06563495399999, 2024-05-03T17:55:07.847 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 24, 2024-05-03T17:55:07.848 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 79, 2024-05-03T17:55:07.848 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 77, 2024-05-03T17:55:07.848 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 185.66546826000001 2024-05-03T17:55:07.848 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-05-03T17:55:07.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 -- 172.21.15.122:0/3111809312 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f089c0b2910 msgr2=0x7f089c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:07.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f089c0b2910 0x7f089c0b4dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f08a0005fd0 tx=0x7f08a0005dc0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 -- 172.21.15.122:0/3111809312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08b01023a0 msgr2=0x7f08b019a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:07.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08b01023a0 0x7f08b019a930 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0060b0 tx=0x7f08ac007f90 comp rx=0 tx=0).stop 2024-05-03T17:55:07.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 -- 172.21.15.122:0/3111809312 shutdown_connections 2024-05-03T17:55:07.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f089c0b2910 0x7f089c0b4dd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b0108410 0x7f08b01a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08b01023a0 0x7f08b019a930 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 --2- 172.21.15.122:0/3111809312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08b0102d60 0x7f08b019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:07.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 -- 172.21.15.122:0/3111809312 >> 172.21.15.122:0/3111809312 conn(0x7f08b0079350 msgr2=0x7f08b0100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:07.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 -- 172.21.15.122:0/3111809312 shutdown_connections 2024-05-03T17:55:07.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:07.845+0000 7f08b8430700 1 -- 172.21.15.122:0/3111809312 wait complete. 2024-05-03T17:55:08.618 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:08 smithi122 ceph-mon[29744]: pgmap v363: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:08.620 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-03T17:55:08.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:08 smithi149 ceph-mon[33409]: pgmap v363: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:08.858 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:08.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:08 smithi192 ceph-mon[33753]: pgmap v363: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:10.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.636+0000 7fa6484e3700 1 -- 172.21.15.122:0/715772336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa640108410 msgr2=0x7fa6401087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:10.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.636+0000 7fa6484e3700 1 --2- 172.21.15.122:0/715772336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa640108410 0x7fa6401087f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa630009990 tx=0x7fa630009ca0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.636+0000 7fa6484e3700 1 -- 172.21.15.122:0/715772336 shutdown_connections 2024-05-03T17:55:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.636+0000 7fa6484e3700 1 --2- 172.21.15.122:0/715772336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa640102d60 0x7fa64010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.636+0000 7fa6484e3700 1 --2- 172.21.15.122:0/715772336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa640108410 0x7fa6401087f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.636+0000 7fa6484e3700 1 --2- 172.21.15.122:0/715772336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6401023a0 0x7fa640102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.636+0000 7fa6484e3700 1 -- 172.21.15.122:0/715772336 >> 172.21.15.122:0/715772336 conn(0x7fa640079350 msgr2=0x7fa640079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.636+0000 7fa6484e3700 1 -- 172.21.15.122:0/715772336 shutdown_connections 2024-05-03T17:55:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.636+0000 7fa6484e3700 1 -- 172.21.15.122:0/715772336 wait complete. 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.637+0000 7fa6484e3700 1 Processor -- start 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.637+0000 7fa6484e3700 1 -- start start 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.637+0000 7fa6484e3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6401023a0 0x7fa64019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.637+0000 7fa6484e3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640102d60 0x7fa64019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.637+0000 7fa6484e3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa640108410 0x7fa640199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.637+0000 7fa6484e3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa640112530 con 0x7fa640108410 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.637+0000 7fa6484e3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6401123b0 con 0x7fa6401023a0 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.637+0000 7fa6484e3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa6401126b0 con 0x7fa640102d60 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.637+0000 7fa646a80700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa640108410 0x7fa640199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.637+0000 7fa646a80700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa640108410 0x7fa640199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48294/0 (socket says 172.21.15.122:48294) 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.637+0000 7fa646a80700 1 -- 172.21.15.122:0/584833516 learned_addr learned my addr 172.21.15.122:0/584833516 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:10.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa645a7e700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640102d60 0x7fa64019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa64627f700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6401023a0 0x7fa64019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa646a80700 1 -- 172.21.15.122:0/584833516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640102d60 msgr2=0x7fa64019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa646a80700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640102d60 0x7fa64019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa646a80700 1 -- 172.21.15.122:0/584833516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6401023a0 msgr2=0x7fa64019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa646a80700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6401023a0 0x7fa64019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa646a80700 1 -- 172.21.15.122:0/584833516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64019a070 con 0x7fa640108410 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa646a80700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa640108410 0x7fa640199910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa638007d70 tx=0x7fa63800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa6377fe700 1 -- 172.21.15.122:0/584833516 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638016020 con 0x7fa640108410 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa6377fe700 1 -- 172.21.15.122:0/584833516 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63800ad40 con 0x7fa640108410 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa6484e3700 1 -- 172.21.15.122:0/584833516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64019a360 con 0x7fa640108410 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa6377fe700 1 -- 172.21.15.122:0/584833516 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6380115e0 con 0x7fa640108410 2024-05-03T17:55:10.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.638+0000 7fa6484e3700 1 -- 172.21.15.122:0/584833516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6401a65d0 con 0x7fa640108410 2024-05-03T17:55:10.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.640+0000 7fa6377fe700 1 -- 172.21.15.122:0/584833516 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa638019020 con 0x7fa640108410 2024-05-03T17:55:10.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.641+0000 7fa6377fe700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa62c0b2910 0x7fa62c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.641+0000 7fa64627f700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa62c0b2910 0x7fa62c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.641+0000 7fa6377fe700 1 -- 172.21.15.122:0/584833516 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fa6380e2110 con 0x7fa640108410 2024-05-03T17:55:10.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.641+0000 7fa652d76b80 1 Processor -- start 2024-05-03T17:55:10.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.641+0000 7fa64627f700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa62c0b2910 0x7fa62c0b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa630006010 tx=0x7fa63001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:10.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa652d76b80 1 -- start start 2024-05-03T17:55:10.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e800c10 0x55650e800ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e805230 0x55650e805690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa652d76b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55650e753810 con 0x55650e801530 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa652d76b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55650e6f2770 con 0x55650e805230 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa652d76b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55650e74f7d0 con 0x55650e800c10 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa645a7e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa645a7e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48308/0 (socket says 172.21.15.122:48308) 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa645a7e700 1 -- 172.21.15.122:0/4114216892 learned_addr learned my addr 172.21.15.122:0/4114216892 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa646a80700 1 --2- 172.21.15.122:0/4114216892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e805230 0x55650e805690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa64627f700 1 --2- 172.21.15.122:0/4114216892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e800c10 0x55650e800ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa645a7e700 1 -- 172.21.15.122:0/4114216892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e800c10 msgr2=0x55650e800ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa645a7e700 1 --2- 172.21.15.122:0/4114216892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e800c10 0x55650e800ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa645a7e700 1 -- 172.21.15.122:0/4114216892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e805230 msgr2=0x55650e805690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa645a7e700 1 --2- 172.21.15.122:0/4114216892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e805230 0x55650e805690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.642+0000 7fa645a7e700 1 -- 172.21.15.122:0/4114216892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650e6a0880 con 0x55650e801530 2024-05-03T17:55:10.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.643+0000 7fa645a7e700 1 --2- 172.21.15.122:0/4114216892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804cf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa63c008c20 tx=0x7fa63c008f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9daa8af7d86b80f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:10.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.643+0000 7fa634d70700 1 -- 172.21.15.122:0/4114216892 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c014070 con 0x55650e801530 2024-05-03T17:55:10.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.643+0000 7fa634d70700 1 -- 172.21.15.122:0/4114216892 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63c004ce0 con 0x55650e801530 2024-05-03T17:55:10.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.643+0000 7fa634d70700 1 -- 172.21.15.122:0/4114216892 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c010430 con 0x55650e801530 2024-05-03T17:55:10.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.643+0000 7fa652d76b80 1 -- 172.21.15.122:0/4114216892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 msgr2=0x55650e804cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:10.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.643+0000 7fa652d76b80 1 --2- 172.21.15.122:0/4114216892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804cf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa63c008c20 tx=0x7fa63c008f30 comp rx=0 tx=0).stop 2024-05-03T17:55:10.641 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:10 smithi122 ceph-mon[29744]: pgmap v364: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.644+0000 7fa652d76b80 1 -- 172.21.15.122:0/4114216892 shutdown_connections 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.644+0000 7fa652d76b80 1 --2- 172.21.15.122:0/4114216892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e805230 0x55650e805690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.644+0000 7fa652d76b80 1 --2- 172.21.15.122:0/4114216892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e800c10 0x55650e800ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.644+0000 7fa652d76b80 1 --2- 172.21.15.122:0/4114216892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804cf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.644+0000 7fa652d76b80 1 -- 172.21.15.122:0/4114216892 >> 172.21.15.122:0/4114216892 conn(0x55650e72d0b0 msgr2=0x55650e7fb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.644+0000 7fa652d76b80 1 -- 172.21.15.122:0/4114216892 shutdown_connections 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.644+0000 7fa652d76b80 1 -- 172.21.15.122:0/4114216892 wait complete. 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.644+0000 7fa652d76b80 1 -- start start 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.644+0000 7fa652d76b80 1 Processor -- start 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e80c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e801530 0x55650e809a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 0x55650e809f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa652d76b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55650e80d490 con 0x55650e805230 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa652d76b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55650e6b9be0 con 0x55650e800c10 2024-05-03T17:55:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa652d76b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55650e80d610 con 0x55650e801530 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa652d76b80 10 client.0 _ll_register_callbacks cb 0x55650e6eb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa64627f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e80c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa646a80700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 0x55650e809f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa645a7e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e801530 0x55650e809a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa64627f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e80c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.122:49818/0 (socket says 172.21.15.122:49818) 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa64627f700 1 -- 172.21.15.122:0/2036350682 learned_addr learned my addr 172.21.15.122:0/2036350682 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa646a80700 1 -- 172.21.15.122:0/2036350682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e801530 msgr2=0x55650e809a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa646a80700 1 --2- 172.21.15.122:0/2036350682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e801530 0x55650e809a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa646a80700 1 -- 172.21.15.122:0/2036350682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 msgr2=0x55650e80c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa646a80700 1 --2- 172.21.15.122:0/2036350682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e80c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa646a80700 1 -- 172.21.15.122:0/2036350682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650e6ef640 con 0x55650e805230 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa646a80700 1 --2- 172.21.15.122:0/2036350682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 0x55650e809f90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa6380dfcd0 tx=0x7fa6380a9d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afa7e009ae4639f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa629ffb700 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa629ffb700 1 -- 172.21.15.122:0/2036350682 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6380c3060 con 0x55650e805230 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa629ffb700 1 -- 172.21.15.122:0/2036350682 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fa6380912f0 con 0x55650e805230 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.645+0000 7fa652d76b80 1 -- 172.21.15.122:0/2036350682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa63c004ce0 con 0x55650e805230 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa629ffb700 1 -- 172.21.15.122:0/2036350682 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa6380a8f50 con 0x55650e805230 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa652d76b80 10 client.15030 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa652d76b80 1 -- 172.21.15.122:0/2036350682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55650e817d60 con 0x55650e805230 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa629ffb700 1 -- 172.21.15.122:0/2036350682 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63c004ce0 con 0x55650e805230 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa629ffb700 1 -- 172.21.15.122:0/2036350682 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa6380aab10 con 0x55650e805230 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa652d76b80 10 client.15030 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa652d76b80 10 client.15030 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa652d76b80 20 client.15030 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2036350682 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x55650e81a360 0x55650e81b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa652d76b80 4 client.15030 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T17:55:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa652d76b80 1 -- 172.21.15.122:0/2036350682 --> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55650e7611f0 con 0x55650e81a360 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.646+0000 7fa64627f700 1 --2- 172.21.15.122:0/2036350682 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x55650e81a360 0x55650e81b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.647+0000 7fa629ffb700 10 client.15030 ms_handle_connect on v2:172.21.15.192:6836/3476407415 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.647+0000 7fa64627f700 1 --2- 172.21.15.122:0/2036350682 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x55650e81a360 0x55650e81b7d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ac7e71b290e9ce server_cookie=81ae19111329427a in_seq=0 out_seq=0 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa629ffb700 1 -- 172.21.15.122:0/2036350682 <== mds.1 v2:172.21.15.192:6836/3476407415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55650e7611f0 con 0x55650e81a360 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa629ffb700 10 client.15030 handle_command_reply: tid=0 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 1 client.15030 shutdown 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 1 -- 172.21.15.122:0/2036350682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 msgr2=0x55650e809f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2036350682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 0x55650e809f90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa6380dfcd0 tx=0x7fa6380a9d20 comp rx=0 tx=0).stop 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 1 -- 172.21.15.122:0/2036350682 shutdown_connections 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2036350682 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x55650e81a360 0x55650e81b7d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2036350682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 0x55650e809f90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2036350682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e80c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa629ffb700 0 client.15030 ms_handle_reset on v2:172.21.15.192:6836/3476407415 2024-05-03T17:55:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2036350682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e801530 0x55650e809a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 1 -- 172.21.15.122:0/2036350682 >> 172.21.15.122:0/2036350682 conn(0x55650e72d0b0 msgr2=0x55650e7fa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 1 -- 172.21.15.122:0/2036350682 shutdown_connections 2024-05-03T17:55:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 1 -- 172.21.15.122:0/2036350682 wait complete. 2024-05-03T17:55:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.648+0000 7fa652d76b80 20 client.15030 trim_cache size 0 max 16384 2024-05-03T17:55:10.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa652d76b80 1 Processor -- start 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa652d76b80 1 -- start start 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e825020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e805230 0x55650e7fb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa652d76b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55650e819b10 con 0x55650e801530 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa652d76b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55650e81a1e0 con 0x55650e800c10 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa652d76b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55650e81a8b0 con 0x55650e805230 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa645a7e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa645a7e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48316/0 (socket says 172.21.15.122:48316) 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa645a7e700 1 -- 172.21.15.122:0/2265430514 learned_addr learned my addr 172.21.15.122:0/2265430514 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa645a7e700 1 -- 172.21.15.122:0/2265430514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e805230 msgr2=0x55650e7fb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:10.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa64627f700 1 --2- 172.21.15.122:0/2265430514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e825020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa646a80700 1 --2- 172.21.15.122:0/2265430514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e805230 0x55650e7fb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa645a7e700 1 --2- 172.21.15.122:0/2265430514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e805230 0x55650e7fb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa645a7e700 1 -- 172.21.15.122:0/2265430514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 msgr2=0x55650e825020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:10.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa645a7e700 1 --2- 172.21.15.122:0/2265430514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e825020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa645a7e700 1 -- 172.21.15.122:0/2265430514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650e80d5c0 con 0x55650e801530 2024-05-03T17:55:10.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa64627f700 1 --2- 172.21.15.122:0/2265430514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e825020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:55:10.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa646a80700 1 --2- 172.21.15.122:0/2265430514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e805230 0x55650e7fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:55:10.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.670+0000 7fa645a7e700 1 --2- 172.21.15.122:0/2265430514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa63c005fd0 tx=0x7fa63c00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15bf7c8d4643e38 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:10.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa62bfff700 1 -- 172.21.15.122:0/2265430514 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c027070 con 0x55650e801530 2024-05-03T17:55:10.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa62bfff700 1 -- 172.21.15.122:0/2265430514 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63c004710 con 0x55650e801530 2024-05-03T17:55:10.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa62bfff700 1 -- 172.21.15.122:0/2265430514 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c014040 con 0x55650e801530 2024-05-03T17:55:10.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa652d76b80 1 -- 172.21.15.122:0/2265430514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 msgr2=0x55650e804370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:10.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2265430514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa63c005fd0 tx=0x7fa63c00ebc0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa652d76b80 1 -- 172.21.15.122:0/2265430514 shutdown_connections 2024-05-03T17:55:10.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2265430514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e805230 0x55650e7fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2265430514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e825020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2265430514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e801530 0x55650e804370 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa652d76b80 1 -- 172.21.15.122:0/2265430514 >> 172.21.15.122:0/2265430514 conn(0x55650e72d0b0 msgr2=0x55650e803c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:10.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa652d76b80 1 -- 172.21.15.122:0/2265430514 shutdown_connections 2024-05-03T17:55:10.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.671+0000 7fa652d76b80 1 -- 172.21.15.122:0/2265430514 wait complete. 2024-05-03T17:55:10.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa652d76b80 1 -- start start 2024-05-03T17:55:10.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa652d76b80 1 Processor -- start 2024-05-03T17:55:10.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e805e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e801530 0x55650e806340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa652d76b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 0x55650e888800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa652d76b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55650e81b650 con 0x55650e805230 2024-05-03T17:55:10.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa652d76b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55650e81af80 con 0x55650e800c10 2024-05-03T17:55:10.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa652d76b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55650e81bd20 con 0x55650e801530 2024-05-03T17:55:10.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa652d76b80 10 client.0 _ll_register_callbacks cb 0x55650e6eb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:10.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa646a80700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 0x55650e888800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa646a80700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 0x55650e888800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48330/0 (socket says 172.21.15.122:48330) 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa645a7e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e801530 0x55650e806340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa646a80700 1 -- 172.21.15.122:0/2378015802 learned_addr learned my addr 172.21.15.122:0/2378015802 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.672+0000 7fa64627f700 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e805e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa646a80700 1 -- 172.21.15.122:0/2378015802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e801530 msgr2=0x55650e806340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa646a80700 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e801530 0x55650e806340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa646a80700 1 -- 172.21.15.122:0/2378015802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 msgr2=0x55650e805e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa646a80700 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e805e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa646a80700 1 -- 172.21.15.122:0/2378015802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650e888d40 con 0x55650e805230 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa64627f700 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e805e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa646a80700 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 0x55650e888800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa63800a740 tx=0x7fa6380a5cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f867ffab0a537 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa629ffb700 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-05-03T17:55:10.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa629ffb700 1 -- 172.21.15.122:0/2378015802 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6380a9ee0 con 0x55650e805230 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa629ffb700 1 -- 172.21.15.122:0/2378015802 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fa6380912f0 con 0x55650e805230 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa652d76b80 1 -- 172.21.15.122:0/2378015802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa63c004710 con 0x55650e805230 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa629ffb700 1 -- 172.21.15.122:0/2378015802 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa6380c3030 con 0x55650e805230 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa652d76b80 10 client.15036 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa652d76b80 1 -- 172.21.15.122:0/2378015802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55650e889f10 con 0x55650e805230 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa629ffb700 1 -- 172.21.15.122:0/2378015802 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63c004710 con 0x55650e805230 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa629ffb700 1 -- 172.21.15.122:0/2378015802 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa6380aab10 con 0x55650e805230 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa652d76b80 10 client.15036 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa652d76b80 10 client.15036 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa652d76b80 20 client.15036 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x55650e81a360 0x55650e88d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.673+0000 7fa652d76b80 4 client.15036 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T17:55:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa652d76b80 1 -- 172.21.15.122:0/2378015802 --> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] -- command(tid 0: {"prefix": "status"}) -- 0x55650e75b290 con 0x55650e81a360 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa64627f700 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x55650e81a360 0x55650e88d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa629ffb700 10 client.15036 ms_handle_connect on v2:172.21.15.192:6836/3476407415 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa64627f700 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x55650e81a360 0x55650e88d640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5b4b91000f47ccb server_cookie=65113133032701a8 in_seq=0 out_seq=0 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa629ffb700 1 -- 172.21.15.122:0/2378015802 <== mds.1 v2:172.21.15.192:6836/3476407415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55650e75b290 con 0x55650e81a360 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa629ffb700 10 client.15036 handle_command_reply: tid=0 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa652d76b80 1 client.15036 shutdown 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa652d76b80 1 -- 172.21.15.122:0/2378015802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 msgr2=0x55650e888800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 0x55650e888800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa63800a740 tx=0x7fa6380a5cf0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa652d76b80 1 -- 172.21.15.122:0/2378015802 shutdown_connections 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x55650e81a360 0x55650e88d640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55650e805230 0x55650e888800 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650e800c10 0x55650e805e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.674+0000 7fa652d76b80 1 --2- 172.21.15.122:0/2378015802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55650e801530 0x55650e806340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa652d76b80 1 -- 172.21.15.122:0/2378015802 >> 172.21.15.122:0/2378015802 conn(0x55650e72d0b0 msgr2=0x55650e8044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:10.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa629ffb700 0 client.15036 ms_handle_reset on v2:172.21.15.192:6836/3476407415 2024-05-03T17:55:10.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa652d76b80 1 -- 172.21.15.122:0/2378015802 shutdown_connections 2024-05-03T17:55:10.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa652d76b80 1 -- 172.21.15.122:0/2378015802 wait complete. 2024-05-03T17:55:10.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa652d76b80 20 client.15036 trim_cache size 0 max 16384 2024-05-03T17:55:10.677 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-05-03T17:55:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "a46fc846-0974-11ef-bc97-c7b262605968", 2024-05-03T17:55:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 1, 2024-05-03T17:55:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24463, 2024-05-03T17:55:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-05-03T17:55:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-05-03T17:55:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-05-03T17:55:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 140.90466071500001, 2024-05-03T17:55:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 24, 2024-05-03T17:55:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 79, 2024-05-03T17:55:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 77, 2024-05-03T17:55:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 185.747218594 2024-05-03T17:55:10.678 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-05-03T17:55:10.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 -- 172.21.15.122:0/584833516 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa62c0b2910 msgr2=0x7fa62c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:10.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa62c0b2910 0x7fa62c0b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa630006010 tx=0x7fa63001a040 comp rx=0 tx=0).stop 2024-05-03T17:55:10.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 -- 172.21.15.122:0/584833516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa640108410 msgr2=0x7fa640199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:10.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa640108410 0x7fa640199910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa638007d70 tx=0x7fa63800a340 comp rx=0 tx=0).stop 2024-05-03T17:55:10.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 -- 172.21.15.122:0/584833516 shutdown_connections 2024-05-03T17:55:10.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fa62c0b2910 0x7fa62c0b4dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa640108410 0x7fa640199910 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6401023a0 0x7fa64019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 --2- 172.21.15.122:0/584833516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640102d60 0x7fa64019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:10.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 -- 172.21.15.122:0/584833516 >> 172.21.15.122:0/584833516 conn(0x7fa640079350 msgr2=0x7fa6401016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:10.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 -- 172.21.15.122:0/584833516 shutdown_connections 2024-05-03T17:55:10.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:10.675+0000 7fa6484e3700 1 -- 172.21.15.122:0/584833516 wait complete. 2024-05-03T17:55:10.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:10 smithi149 ceph-mon[33409]: pgmap v364: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:10.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:10 smithi192 ceph-mon[33753]: pgmap v364: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:11.448 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-03T17:55:11.696 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:12.691 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:12 smithi122 ceph-mon[29744]: pgmap v365: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:12.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:12 smithi149 ceph-mon[33409]: pgmap v365: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:12.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:12 smithi192 ceph-mon[33753]: pgmap v365: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:13.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.347+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/3679319860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfc8108410 msgr2=0x7fbfc81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:13.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.347+0000 7fbfcdd2f700 1 --2- 172.21.15.122:0/3679319860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfc8108410 0x7fbfc81087f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b0a0 tx=0x7fbfc000b3b0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.347+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/3679319860 shutdown_connections 2024-05-03T17:55:13.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.347+0000 7fbfcdd2f700 1 --2- 172.21.15.122:0/3679319860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfc8102d60 0x7fbfc810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.347+0000 7fbfcdd2f700 1 --2- 172.21.15.122:0/3679319860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfc8108410 0x7fbfc81087f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.347+0000 7fbfcdd2f700 1 --2- 172.21.15.122:0/3679319860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc81023a0 0x7fbfc8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.347+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/3679319860 >> 172.21.15.122:0/3679319860 conn(0x7fbfc8079350 msgr2=0x7fbfc8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.347+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/3679319860 shutdown_connections 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.347+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/3679319860 wait complete. 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfcdd2f700 1 Processor -- start 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfcdd2f700 1 -- start start 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfcdd2f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfc81023a0 0x7fbfc819f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfcdd2f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfc8102d60 0x7fbfc819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfcdd2f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc8108410 0x7fbfc8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfcdd2f700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc8113730 con 0x7fbfc8102d60 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfcdd2f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81135b0 con 0x7fbfc8108410 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfcdd2f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81138b0 con 0x7fbfc81023a0 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfc8102d60 0x7fbfc819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfc8102d60 0x7fbfc819fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48356/0 (socket says 172.21.15.122:48356) 2024-05-03T17:55:13.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfc6ffd700 1 -- 172.21.15.122:0/4213848266 learned_addr learned my addr 172.21.15.122:0/4213848266 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:13.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfc7fff700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc8108410 0x7fbfc8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfc81023a0 0x7fbfc819f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfc6ffd700 1 -- 172.21.15.122:0/4213848266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfc81023a0 msgr2=0x7fbfc819f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:13.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfc81023a0 0x7fbfc819f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfc6ffd700 1 -- 172.21.15.122:0/4213848266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc8108410 msgr2=0x7fbfc8199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:13.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc8108410 0x7fbfc8199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.348+0000 7fbfc6ffd700 1 -- 172.21.15.122:0/4213848266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc819a290 con 0x7fbfc8102d60 2024-05-03T17:55:13.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.349+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfc8102d60 0x7fbfc819fe60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000b530 tx=0x7fbfb000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:13.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.349+0000 7fbfc4ff9700 1 -- 172.21.15.122:0/4213848266 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0015020 con 0x7fbfc8102d60 2024-05-03T17:55:13.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.349+0000 7fbfc4ff9700 1 -- 172.21.15.122:0/4213848266 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb0011ce0 con 0x7fbfc8102d60 2024-05-03T17:55:13.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.349+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/4213848266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc819a580 con 0x7fbfc8102d60 2024-05-03T17:55:13.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.349+0000 7fbfc4ff9700 1 -- 172.21.15.122:0/4213848266 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0010e40 con 0x7fbfc8102d60 2024-05-03T17:55:13.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.349+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/4213848266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc81a67f0 con 0x7fbfc8102d60 2024-05-03T17:55:13.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.351+0000 7fbfc4ff9700 1 -- 172.21.15.122:0/4213848266 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fbfb0018020 con 0x7fbfc8102d60 2024-05-03T17:55:13.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.352+0000 7fbfc4ff9700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbfb40b2910 0x7fbfb40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.352+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbfb40b2910 0x7fbfb40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.352+0000 7fbfc4ff9700 1 -- 172.21.15.122:0/4213848266 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fbfb00e19c0 con 0x7fbfc8102d60 2024-05-03T17:55:13.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.352+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbfb40b2910 0x7fbfb40b4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0006130 tx=0x7fbfc0006080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:13.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.352+0000 7fbfd85c2b80 1 Processor -- start 2024-05-03T17:55:13.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.352+0000 7fbfd85c2b80 1 -- start start 2024-05-03T17:55:13.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f5589e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f55898350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 0x563f55898d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563f557e68f0 con 0x563f5589dd00 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563f5574bd50 con 0x563f558988e0 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563f557e28b0 con 0x563f5589e620 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f5589e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f5589e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48368/0 (socket says 172.21.15.122:48368) 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc77fe700 1 -- 172.21.15.122:0/720105637 learned_addr learned my addr 172.21.15.122:0/720105637 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/720105637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f55898350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc77fe700 1 -- 172.21.15.122:0/720105637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 msgr2=0x563f55898350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/720105637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f55898350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc77fe700 1 -- 172.21.15.122:0/720105637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 msgr2=0x563f55898d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc7fff700 1 --2- 172.21.15.122:0/720105637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 0x563f55898d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/720105637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 0x563f55898d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc77fe700 1 -- 172.21.15.122:0/720105637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f55733880 con 0x563f5589dd00 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/720105637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f55898350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc7fff700 1 --2- 172.21.15.122:0/720105637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 0x563f55898d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/720105637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f5589e0e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00092a0 tx=0x7fbfc001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddbe95529faa0af server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfbdffb700 1 -- 172.21.15.122:0/720105637 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0007bf0 con 0x563f5589dd00 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfbdffb700 1 -- 172.21.15.122:0/720105637 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc002d070 con 0x563f5589dd00 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.353+0000 7fbfbdffb700 1 -- 172.21.15.122:0/720105637 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0005240 con 0x563f5589dd00 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.354+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/720105637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 msgr2=0x563f5589e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.354+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/720105637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f5589e0e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00092a0 tx=0x7fbfc001c000 comp rx=0 tx=0).stop 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.354+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/720105637 shutdown_connections 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.354+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/720105637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 0x563f55898d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.354+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/720105637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f5589e0e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.354+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/720105637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f55898350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.354+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/720105637 >> 172.21.15.122:0/720105637 conn(0x563f557c0b20 msgr2=0x563f5588e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.354+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/720105637 shutdown_connections 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.354+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/720105637 wait complete. 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfd85c2b80 1 -- start start 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfd85c2b80 1 Processor -- start 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f558a4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5589dd00 0x563f558a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f558a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563f558a4ff0 con 0x563f558988e0 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563f5579fe80 con 0x563f5589dd00 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563f558a5170 con 0x563f5589e620 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfd85c2b80 10 client.0 _ll_register_callbacks cb 0x563f5577a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f558a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f558a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48376/0 (socket says 172.21.15.122:48376) 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfc77fe700 1 -- 172.21.15.122:0/223852233 learned_addr learned my addr 172.21.15.122:0/223852233 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5589dd00 0x563f558a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfc7fff700 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f558a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfc77fe700 1 -- 172.21.15.122:0/223852233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 msgr2=0x563f558a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f558a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfc77fe700 1 -- 172.21.15.122:0/223852233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5589dd00 msgr2=0x563f558a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5589dd00 0x563f558a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.355+0000 7fbfc77fe700 1 -- 172.21.15.122:0/223852233 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5579d710 con 0x563f558988e0 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f558a4200 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b3f0 tx=0x7fbfc00092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692cdfd55d84a858 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfa6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/223852233 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0035070 con 0x563f558988e0 2024-05-03T17:55:13.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/223852233 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fbfc001c990 con 0x563f558988e0 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/223852233 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc002d070 con 0x563f558988e0 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/223852233 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbfc001cb90 con 0x563f558988e0 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfd85c2b80 10 client.15048 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/223852233 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563f558af740 con 0x563f558988e0 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/223852233 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc002d070 con 0x563f558988e0 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/223852233 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbfc0011070 con 0x563f558988e0 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfd85c2b80 10 client.15048 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfd85c2b80 10 client.15048 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfd85c2b80 20 client.15048 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x563f558b0a90 0x563f558b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfd85c2b80 4 client.15048 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.356+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/223852233 --> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f557f42d0 con 0x563f558b0a90 2024-05-03T17:55:13.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.357+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x563f558b0a90 0x563f558b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.357+0000 7fbfa6ffd700 10 client.15048 ms_handle_connect on v2:172.21.15.192:6838/1429818440 2024-05-03T17:55:13.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.357+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x563f558b0a90 0x563f558b2c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30399ba56b4d631b server_cookie=98ef38bd9f320957 in_seq=0 out_seq=0 2024-05-03T17:55:13.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/223852233 <== mds.0 v2:172.21.15.192:6838/1429818440 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f557f42d0 con 0x563f558b0a90 2024-05-03T17:55:13.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfa6ffd700 10 client.15048 handle_command_reply: tid=0 2024-05-03T17:55:13.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 1 client.15048 shutdown 2024-05-03T17:55:13.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/223852233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 msgr2=0x563f558a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:13.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f558a4200 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b3f0 tx=0x7fbfc00092d0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/223852233 shutdown_connections 2024-05-03T17:55:13.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f558a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x563f558b0a90 0x563f558b2c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f558a4200 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/223852233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5589dd00 0x563f558a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/223852233 >> 172.21.15.122:0/223852233 conn(0x563f557c0b20 msgr2=0x563f5589d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:13.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfa6ffd700 0 client.15048 ms_handle_reset on v2:172.21.15.192:6838/1429818440 2024-05-03T17:55:13.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/223852233 shutdown_connections 2024-05-03T17:55:13.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/223852233 wait complete. 2024-05-03T17:55:13.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.358+0000 7fbfd85c2b80 20 client.15048 trim_cache size 0 max 16384 2024-05-03T17:55:13.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfd85c2b80 1 Processor -- start 2024-05-03T17:55:13.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfd85c2b80 1 -- start start 2024-05-03T17:55:13.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f5589f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5589dd00 0x563f55890420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f5588e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563f558b0f70 con 0x563f558988e0 2024-05-03T17:55:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563f558b1640 con 0x563f5589dd00 2024-05-03T17:55:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563f558b1d10 con 0x563f5589e620 2024-05-03T17:55:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f5589f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f5589f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48384/0 (socket says 172.21.15.122:48384) 2024-05-03T17:55:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfc77fe700 1 -- 172.21.15.122:0/2424148286 learned_addr learned my addr 172.21.15.122:0/2424148286 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/2424148286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5589dd00 0x563f55890420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.379+0000 7fbfc7fff700 1 --2- 172.21.15.122:0/2424148286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f5588e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.380+0000 7fbfc77fe700 1 -- 172.21.15.122:0/2424148286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 msgr2=0x563f5588e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.380+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/2424148286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f5588e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.380+0000 7fbfc77fe700 1 -- 172.21.15.122:0/2424148286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5589dd00 msgr2=0x563f55890420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.380+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/2424148286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5589dd00 0x563f55890420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.380+0000 7fbfc77fe700 1 -- 172.21.15.122:0/2424148286 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f55889730 con 0x563f558988e0 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.380+0000 7fbfc7fff700 1 --2- 172.21.15.122:0/2424148286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f5588e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.380+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/2424148286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f5589f290 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbfc001cfd0 tx=0x7fbfc0009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c4438fcd51a570 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.380+0000 7fbfbd7fa700 1 -- 172.21.15.122:0/2424148286 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0035030 con 0x563f558988e0 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.380+0000 7fbfbd7fa700 1 -- 172.21.15.122:0/2424148286 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc002d070 con 0x563f558988e0 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.380+0000 7fbfbd7fa700 1 -- 172.21.15.122:0/2424148286 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc003a9f0 con 0x563f558988e0 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2424148286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 msgr2=0x563f5589f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/2424148286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f5589f290 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbfc001cfd0 tx=0x7fbfc0009340 comp rx=0 tx=0).stop 2024-05-03T17:55:13.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2424148286 shutdown_connections 2024-05-03T17:55:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/2424148286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f5588e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/2424148286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f558988e0 0x563f5589f290 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/2424148286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5589dd00 0x563f55890420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2424148286 >> 172.21.15.122:0/2424148286 conn(0x563f557c0b20 msgr2=0x563f5588b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2424148286 shutdown_connections 2024-05-03T17:55:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2424148286 wait complete. 2024-05-03T17:55:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 -- start start 2024-05-03T17:55:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 Processor -- start 2024-05-03T17:55:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 0x563f5589ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f558bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f558bc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563f558b2ab0 con 0x563f5589dd00 2024-05-03T17:55:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563f558b23e0 con 0x563f558988e0 2024-05-03T17:55:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563f558b3180 con 0x563f5589e620 2024-05-03T17:55:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.381+0000 7fbfd85c2b80 10 client.0 _ll_register_callbacks cb 0x563f5577a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f558bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f558bc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48398/0 (socket says 172.21.15.122:48398) 2024-05-03T17:55:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc6ffd700 1 -- 172.21.15.122:0/2915182699 learned_addr learned my addr 172.21.15.122:0/2915182699 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc7fff700 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f558bc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 0x563f5589ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc6ffd700 1 -- 172.21.15.122:0/2915182699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 msgr2=0x563f558bc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:13.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f558bc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc6ffd700 1 -- 172.21.15.122:0/2915182699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 msgr2=0x563f5589ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:13.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 0x563f5589ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc6ffd700 1 -- 172.21.15.122:0/2915182699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f558a2bd0 con 0x563f5589dd00 2024-05-03T17:55:13.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc7fff700 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f558bc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:55:13.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 0x563f5589ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:55:13.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfc6ffd700 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f558bc0a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00a5110 tx=0x7fbfb001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10219c57dceea78 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:13.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfa6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-05-03T17:55:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/2915182699 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00a9500 con 0x563f5589dd00 2024-05-03T17:55:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/2915182699 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fbfb00b1a60 con 0x563f5589dd00 2024-05-03T17:55:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2915182699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc002d070 con 0x563f5589dd00 2024-05-03T17:55:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/2915182699 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbfb00b1c60 con 0x563f5589dd00 2024-05-03T17:55:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfd85c2b80 10 client.15054 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2915182699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563f5591d150 con 0x563f5589dd00 2024-05-03T17:55:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.382+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/2915182699 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc002d070 con 0x563f5589dd00 2024-05-03T17:55:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.383+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/2915182699 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbfb00a4a00 con 0x563f5589dd00 2024-05-03T17:55:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.383+0000 7fbfd85c2b80 10 client.15054 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.383+0000 7fbfd85c2b80 10 client.15054 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T17:55:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.383+0000 7fbfd85c2b80 20 client.15054 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.383+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x563f55921d40 0x563f55920a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.383+0000 7fbfd85c2b80 4 client.15054 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T17:55:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.383+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2915182699 --> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] -- command(tid 0: {"prefix": "status"}) -- 0x563f557ee370 con 0x563f55921d40 2024-05-03T17:55:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.383+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x563f55921d40 0x563f55920a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.383+0000 7fbfa6ffd700 10 client.15054 ms_handle_connect on v2:172.21.15.192:6838/1429818440 2024-05-03T17:55:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.383+0000 7fbfc77fe700 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x563f55921d40 0x563f55920a50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=155135f338158a68 server_cookie=32ca7d1dcc8f3c12 in_seq=0 out_seq=0 2024-05-03T17:55:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfa6ffd700 1 -- 172.21.15.122:0/2915182699 <== mds.0 v2:172.21.15.192:6838/1429818440 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563f557ee370 con 0x563f55921d40 2024-05-03T17:55:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfa6ffd700 10 client.15054 handle_command_reply: tid=0 2024-05-03T17:55:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 1 client.15054 shutdown 2024-05-03T17:55:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2915182699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 msgr2=0x563f558bc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f558bc0a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00a5110 tx=0x7fbfb001c040 comp rx=0 tx=0).stop 2024-05-03T17:55:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2915182699 shutdown_connections 2024-05-03T17:55:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563f5589e620 0x563f558bc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f558988e0 0x563f5589ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x563f55921d40 0x563f55920a50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 1 --2- 172.21.15.122:0/2915182699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f5589dd00 0x563f558bc0a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2915182699 >> 172.21.15.122:0/2915182699 conn(0x563f557c0b20 msgr2=0x563f5588e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfa6ffd700 0 client.15054 ms_handle_reset on v2:172.21.15.192:6838/1429818440 2024-05-03T17:55:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2915182699 shutdown_connections 2024-05-03T17:55:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 1 -- 172.21.15.122:0/2915182699 wait complete. 2024-05-03T17:55:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfd85c2b80 20 client.15054 trim_cache size 0 max 16384 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "a46fc846-0974-11ef-bc97-c7b262605968", 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24481, 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 155.67114088100001, 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 24, 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 79, 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 77, 2024-05-03T17:55:13.389 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 182.791210686 2024-05-03T17:55:13.390 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-05-03T17:55:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/4213848266 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbfb40b2910 msgr2=0x7fbfb40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfcdd2f700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbfb40b2910 0x7fbfb40b4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0006130 tx=0x7fbfc0006080 comp rx=0 tx=0).stop 2024-05-03T17:55:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/4213848266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfc8102d60 msgr2=0x7fbfc819fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.384+0000 7fbfcdd2f700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfc8102d60 0x7fbfc819fe60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000b530 tx=0x7fbfb000b840 comp rx=0 tx=0).stop 2024-05-03T17:55:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.385+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/4213848266 shutdown_connections 2024-05-03T17:55:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.385+0000 7fbfcdd2f700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbfb40b2910 0x7fbfb40b4dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.385+0000 7fbfcdd2f700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc8108410 0x7fbfc8199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.385+0000 7fbfcdd2f700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfc81023a0 0x7fbfc819f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.385+0000 7fbfcdd2f700 1 --2- 172.21.15.122:0/4213848266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfc8102d60 0x7fbfc819fe60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.385+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/4213848266 >> 172.21.15.122:0/4213848266 conn(0x7fbfc8079350 msgr2=0x7fbfc8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.385+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/4213848266 shutdown_connections 2024-05-03T17:55:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:13.385+0000 7fbfcdd2f700 1 -- 172.21.15.122:0/4213848266 wait complete. 2024-05-03T17:55:14.202 INFO:tasks.cephadm:Setup complete, yielding 2024-05-03T17:55:14.203 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-03T17:55:14.212 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-05-03T17:55:14.212 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-03T17:55:14.455 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:14.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:14 smithi122 ceph-mon[29744]: pgmap v366: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:14.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:14 smithi149 ceph-mon[33409]: pgmap v366: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:14.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:14 smithi192 ceph-mon[33753]: pgmap v366: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:15.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.998+0000 7f7b17792700 1 -- 172.21.15.122:0/2236742965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b10103500 msgr2=0x7f7b10103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:15.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.998+0000 7f7b17792700 1 --2- 172.21.15.122:0/2236742965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b10103500 0x7f7b10103920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7b000099c0 tx=0x7f7b00009cd0 comp rx=0 tx=0).stop 2024-05-03T17:55:15.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.998+0000 7f7b17792700 1 -- 172.21.15.122:0/2236742965 shutdown_connections 2024-05-03T17:55:15.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.998+0000 7f7b17792700 1 --2- 172.21.15.122:0/2236742965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b101050a0 0x7f7b1010b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:15.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.998+0000 7f7b17792700 1 --2- 172.21.15.122:0/2236742965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b10103500 0x7f7b10103920 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:15.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.998+0000 7f7b17792700 1 --2- 172.21.15.122:0/2236742965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b10104700 0x7f7b10104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:15.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.998+0000 7f7b17792700 1 -- 172.21.15.122:0/2236742965 >> 172.21.15.122:0/2236742965 conn(0x7f7b100feaa0 msgr2=0x7f7b10100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:15.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.998+0000 7f7b17792700 1 -- 172.21.15.122:0/2236742965 shutdown_connections 2024-05-03T17:55:15.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.998+0000 7f7b17792700 1 -- 172.21.15.122:0/2236742965 wait complete. 2024-05-03T17:55:15.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b17792700 1 Processor -- start 2024-05-03T17:55:15.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b17792700 1 -- start start 2024-05-03T17:55:15.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b17792700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b10103500 0x7f7b10077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:15.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b17792700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b10104700 0x7f7b10075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b17792700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b101050a0 0x7f7b10075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b17792700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010f070 con 0x7f7b10103500 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b17792700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010eef0 con 0x7f7b101050a0 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b17792700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010f1f0 con 0x7f7b10104700 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b1552e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b10103500 0x7f7b10077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b1552e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b10103500 0x7f7b10077240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48414/0 (socket says 172.21.15.122:48414) 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b1552e700 1 -- 172.21.15.122:0/2486409007 learned_addr learned my addr 172.21.15.122:0/2486409007 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b15d2f700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b101050a0 0x7f7b10075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:15.999+0000 7f7b14d2d700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b10104700 0x7f7b10075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.000+0000 7f7b1552e700 1 -- 172.21.15.122:0/2486409007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b10104700 msgr2=0x7f7b10075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.000+0000 7f7b1552e700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b10104700 0x7f7b10075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.000+0000 7f7b1552e700 1 -- 172.21.15.122:0/2486409007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b101050a0 msgr2=0x7f7b10075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:15.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.000+0000 7f7b1552e700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b101050a0 0x7f7b10075dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:15.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.000+0000 7f7b1552e700 1 -- 172.21.15.122:0/2486409007 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b10076680 con 0x7f7b10103500 2024-05-03T17:55:15.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.000+0000 7f7b1552e700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b10103500 0x7f7b10077240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009f50 tx=0x7f7b000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:15.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.000+0000 7f7b067fc700 1 -- 172.21.15.122:0/2486409007 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00027070 con 0x7f7b10103500 2024-05-03T17:55:15.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.000+0000 7f7b17792700 1 -- 172.21.15.122:0/2486409007 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b101a3870 con 0x7f7b10103500 2024-05-03T17:55:15.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.000+0000 7f7b067fc700 1 -- 172.21.15.122:0/2486409007 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b00022470 con 0x7f7b10103500 2024-05-03T17:55:15.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.000+0000 7f7b067fc700 1 -- 172.21.15.122:0/2486409007 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000f6c0 con 0x7f7b10103500 2024-05-03T17:55:15.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.000+0000 7f7b17792700 1 -- 172.21.15.122:0/2486409007 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b101a3da0 con 0x7f7b10103500 2024-05-03T17:55:15.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.001+0000 7f7b17792700 1 -- 172.21.15.122:0/2486409007 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b1006a190 con 0x7f7b10103500 2024-05-03T17:55:15.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.002+0000 7f7b067fc700 1 -- 172.21.15.122:0/2486409007 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f7b0002a070 con 0x7f7b10103500 2024-05-03T17:55:15.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.003+0000 7f7b067fc700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f7afc0b2910 0x7f7afc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:15.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.003+0000 7f7b14d2d700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f7afc0b2910 0x7f7afc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:16.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.003+0000 7f7b067fc700 1 -- 172.21.15.122:0/2486409007 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f7b000e3610 con 0x7f7b10103500 2024-05-03T17:55:16.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.003+0000 7f7b14d2d700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f7afc0b2910 0x7f7afc0b4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c006fd0 tx=0x7f7b0c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:16.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.006+0000 7f7b067fc700 1 -- 172.21.15.122:0/2486409007 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7b00013070 con 0x7f7b10103500 2024-05-03T17:55:16.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.228+0000 7f7b17792700 1 -- 172.21.15.122:0/2486409007 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7b100638e0 con 0x7f7afc0b2910 2024-05-03T17:55:16.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.229+0000 7f7b067fc700 1 -- 172.21.15.122:0/2486409007 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7b100638e0 con 0x7f7afc0b2910 2024-05-03T17:55:16.224 INFO:teuthology.orchestra.run.smithi122.stdout:Backend: cephadm 2024-05-03T17:55:16.224 INFO:teuthology.orchestra.run.smithi122.stdout:Available: Yes 2024-05-03T17:55:16.224 INFO:teuthology.orchestra.run.smithi122.stdout:Paused: No 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.231+0000 7f7b17792700 1 -- 172.21.15.122:0/2486409007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f7afc0b2910 msgr2=0x7f7afc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.231+0000 7f7b17792700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f7afc0b2910 0x7f7afc0b4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c006fd0 tx=0x7f7b0c008040 comp rx=0 tx=0).stop 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.232+0000 7f7b17792700 1 -- 172.21.15.122:0/2486409007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b10103500 msgr2=0x7f7b10077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.232+0000 7f7b17792700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b10103500 0x7f7b10077240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009f50 tx=0x7f7b000047b0 comp rx=0 tx=0).stop 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.232+0000 7f7b17792700 1 -- 172.21.15.122:0/2486409007 shutdown_connections 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.232+0000 7f7b17792700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f7afc0b2910 0x7f7afc0b4dd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.232+0000 7f7b17792700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b101050a0 0x7f7b10075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.232+0000 7f7b17792700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b10103500 0x7f7b10077240 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.232+0000 7f7b17792700 1 --2- 172.21.15.122:0/2486409007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b10104700 0x7f7b10075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.232+0000 7f7b17792700 1 -- 172.21.15.122:0/2486409007 >> 172.21.15.122:0/2486409007 conn(0x7f7b100feaa0 msgr2=0x7f7b10107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.232+0000 7f7b17792700 1 -- 172.21.15.122:0/2486409007 shutdown_connections 2024-05-03T17:55:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:16.232+0000 7f7b17792700 1 -- 172.21.15.122:0/2486409007 wait complete. 2024-05-03T17:55:16.539 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:16 smithi122 ceph-mon[29744]: pgmap v367: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:16.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:16 smithi149 ceph-mon[33409]: pgmap v367: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:16.841 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-03T17:55:16.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:16 smithi192 ceph-mon[33753]: pgmap v367: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:17.077 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:17.452 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:17 smithi122 ceph-mon[29744]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:17.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:17 smithi149 ceph-mon[33409]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:17.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:17 smithi192 ceph-mon[33753]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:18.500 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:18 smithi122 ceph-mon[29744]: pgmap v368: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:18.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:18 smithi149 ceph-mon[33409]: pgmap v368: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:18.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.867+0000 7ff9fc226700 1 -- 172.21.15.122:0/2510369651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9f41091f0 msgr2=0x7ff9f410b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:18.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.867+0000 7ff9fc226700 1 --2- 172.21.15.122:0/2510369651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9f41091f0 0x7ff9f410b660 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00b0a0 tx=0x7ff9ec00b3b0 comp rx=0 tx=0).stop 2024-05-03T17:55:18.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.867+0000 7ff9fc226700 1 -- 172.21.15.122:0/2510369651 shutdown_connections 2024-05-03T17:55:18.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.867+0000 7ff9fc226700 1 --2- 172.21.15.122:0/2510369651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9f41091f0 0x7ff9f410b660 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:18.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.867+0000 7ff9fc226700 1 --2- 172.21.15.122:0/2510369651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f4103f90 0x7ff9f4106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:18.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.867+0000 7ff9fc226700 1 --2- 172.21.15.122:0/2510369651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9f41068c0 0x7ff9f4108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:18.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.867+0000 7ff9fc226700 1 -- 172.21.15.122:0/2510369651 >> 172.21.15.122:0/2510369651 conn(0x7ff9f40fd9c0 msgr2=0x7ff9f40ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9fc226700 1 -- 172.21.15.122:0/2510369651 shutdown_connections 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9fc226700 1 -- 172.21.15.122:0/2510369651 wait complete. 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9fc226700 1 Processor -- start 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9fc226700 1 -- start start 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9fc226700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f4103f90 0x7ff9f419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9fc226700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9f41068c0 0x7ff9f419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9fc226700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9f41091f0 0x7ff9f41a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9fc226700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f410ef60 con 0x7ff9f41068c0 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9fc226700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f410ede0 con 0x7ff9f4103f90 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9fc226700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f410f0e0 con 0x7ff9f41091f0 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9f97c1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9f41068c0 0x7ff9f419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9f97c1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9f41068c0 0x7ff9f419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57894/0 (socket says 172.21.15.122:57894) 2024-05-03T17:55:18.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.868+0000 7ff9f97c1700 1 -- 172.21.15.122:0/3041770787 learned_addr learned my addr 172.21.15.122:0/3041770787 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9f9fc2700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f4103f90 0x7ff9f419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9fa7c3700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9f41091f0 0x7ff9f41a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9f97c1700 1 -- 172.21.15.122:0/3041770787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9f41091f0 msgr2=0x7ff9f41a2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9f97c1700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9f41091f0 0x7ff9f41a2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9f97c1700 1 -- 172.21.15.122:0/3041770787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f4103f90 msgr2=0x7ff9f419b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9f97c1700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f4103f90 0x7ff9f419b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9f97c1700 1 -- 172.21.15.122:0/3041770787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f41a3380 con 0x7ff9f41068c0 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9f97c1700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9f41068c0 0x7ff9f419bc60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400d980 tx=0x7ff9e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9eaffd700 1 -- 172.21.15.122:0/3041770787 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4016070 con 0x7ff9f41068c0 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9eaffd700 1 -- 172.21.15.122:0/3041770787 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e400de70 con 0x7ff9f41068c0 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9fc226700 1 -- 172.21.15.122:0/3041770787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f41a3670 con 0x7ff9f41068c0 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9eaffd700 1 -- 172.21.15.122:0/3041770787 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400f3c0 con 0x7ff9f41068c0 2024-05-03T17:55:18.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.869+0000 7ff9fc226700 1 -- 172.21.15.122:0/3041770787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f41a3bb0 con 0x7ff9f41068c0 2024-05-03T17:55:18.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:18 smithi192 ceph-mon[33753]: pgmap v368: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:18.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.870+0000 7ff9fc226700 1 -- 172.21.15.122:0/3041770787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f406a190 con 0x7ff9f41068c0 2024-05-03T17:55:18.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.871+0000 7ff9eaffd700 1 -- 172.21.15.122:0/3041770787 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7ff9e4019070 con 0x7ff9f41068c0 2024-05-03T17:55:18.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.872+0000 7ff9eaffd700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff9e00b2910 0x7ff9e00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:18.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.872+0000 7ff9f9fc2700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff9e00b2910 0x7ff9e00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:18.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.872+0000 7ff9eaffd700 1 -- 172.21.15.122:0/3041770787 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7ff9e40e1dd0 con 0x7ff9f41068c0 2024-05-03T17:55:18.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.872+0000 7ff9f9fc2700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff9e00b2910 0x7ff9e00b4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000ac50 tx=0x7ff9f000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:18.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:18.875+0000 7ff9eaffd700 1 -- 172.21.15.122:0/3041770787 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff9e40a9a90 con 0x7ff9f41068c0 2024-05-03T17:55:19.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.098+0000 7ff9fc226700 1 -- 172.21.15.122:0/3041770787 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff9f40638e0 con 0x7ff9e00b2910 2024-05-03T17:55:19.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.112+0000 7ff9eaffd700 1 -- 172.21.15.122:0/3041770787 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7ff9f40638e0 con 0x7ff9e00b2910 2024-05-03T17:55:19.107 INFO:teuthology.orchestra.run.smithi122.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-03T17:55:19.107 INFO:teuthology.orchestra.run.smithi122.stdout:mds.a smithi122 running (3m) 30s ago 3m 15.9M - 19.0.0-3450-g9a047fe7 ee4d84620249 818cdb0c4598 2024-05-03T17:55:19.107 INFO:teuthology.orchestra.run.smithi122.stdout:mds.b smithi149 running (2m) 44s ago 2m 20.0M - 19.0.0-3450-g9a047fe7 ee4d84620249 4c322e9da4d4 2024-05-03T17:55:19.107 INFO:teuthology.orchestra.run.smithi122.stdout:mds.c smithi192 running (3m) 44s ago 3m 18.7M - 19.0.0-3450-g9a047fe7 ee4d84620249 d4bd65c0a165 2024-05-03T17:55:19.107 INFO:teuthology.orchestra.run.smithi122.stdout:mds.d smithi122 running (3m) 30s ago 3m 17.0M - 19.0.0-3450-g9a047fe7 ee4d84620249 53dcd52b8677 2024-05-03T17:55:19.107 INFO:teuthology.orchestra.run.smithi122.stdout:mds.e smithi149 running (3m) 44s ago 3m 14.5M - 19.0.0-3450-g9a047fe7 ee4d84620249 69442177f39d 2024-05-03T17:55:19.107 INFO:teuthology.orchestra.run.smithi122.stdout:mds.f smithi192 running (3m) 44s ago 3m 20.7M - 19.0.0-3450-g9a047fe7 ee4d84620249 81729ac8f345 2024-05-03T17:55:19.107 INFO:teuthology.orchestra.run.smithi122.stdout:mds.g smithi122 running (3m) 30s ago 3m 16.7M - 19.0.0-3450-g9a047fe7 ee4d84620249 c5e8610b8cf3 2024-05-03T17:55:19.107 INFO:teuthology.orchestra.run.smithi122.stdout:mds.h smithi149 running (3m) 44s ago 3m 20.5M - 19.0.0-3450-g9a047fe7 ee4d84620249 d43d175fadb9 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:mds.i smithi192 running (3m) 44s ago 3m 26.5M - 19.0.0-3450-g9a047fe7 ee4d84620249 b336f769e1d5 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:mds.j smithi122 running (3m) 30s ago 3m 14.6M - 19.0.0-3450-g9a047fe7 ee4d84620249 91076c7dc012 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:mds.k smithi149 running (3m) 44s ago 3m 14.5M - 19.0.0-3450-g9a047fe7 ee4d84620249 4a1560d17b8a 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:mds.l smithi192 running (3m) 44s ago 3m 18.4M - 19.0.0-3450-g9a047fe7 ee4d84620249 67a1d0769811 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.x smithi122 *:9283,8765 running (10m) 30s ago 10m 531M - 19.0.0-3450-g9a047fe7 ee4d84620249 fec6baba9e62 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.y smithi149 *:8443,8765 running (8m) 44s ago 8m 466M - 19.0.0-3450-g9a047fe7 ee4d84620249 47a31b126353 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.z smithi192 *:8443,8765 running (8m) 44s ago 8m 466M - 19.0.0-3450-g9a047fe7 ee4d84620249 a1483c20e32d 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:mon.a smithi122 running (10m) 30s ago 11m 51.2M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 6c70458e7d5b 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:mon.b smithi149 running (9m) 44s ago 9m 38.4M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 35163dfde530 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:mon.c smithi192 running (9m) 44s ago 9m 43.3M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 cac55d075388 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 smithi122 running (8m) 30s ago 8m 57.8M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 209ca2f75e93 2024-05-03T17:55:19.108 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 smithi149 running (8m) 44s ago 8m 55.4M 1454M 19.0.0-3450-g9a047fe7 ee4d84620249 44041540fe9f 2024-05-03T17:55:19.109 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 smithi192 running (7m) 44s ago 7m 56.4M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 e41c69e15eae 2024-05-03T17:55:19.109 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 smithi122 running (7m) 30s ago 7m 43.9M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 d05a30e9caf0 2024-05-03T17:55:19.109 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 smithi149 running (6m) 44s ago 6m 51.6M 1454M 19.0.0-3450-g9a047fe7 ee4d84620249 f658c8da154a 2024-05-03T17:55:19.109 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 smithi192 running (6m) 44s ago 6m 54.0M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 d8886656ad3d 2024-05-03T17:55:19.109 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 smithi122 running (6m) 30s ago 6m 52.5M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 8b4924e76ff8 2024-05-03T17:55:19.109 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 smithi149 running (5m) 44s ago 5m 53.9M 1454M 19.0.0-3450-g9a047fe7 ee4d84620249 417c7c8d1e2e 2024-05-03T17:55:19.109 INFO:teuthology.orchestra.run.smithi122.stdout:osd.8 smithi192 running (5m) 44s ago 5m 55.2M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 76ca6f555f23 2024-05-03T17:55:19.109 INFO:teuthology.orchestra.run.smithi122.stdout:osd.9 smithi122 running (4m) 30s ago 4m 53.4M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 be4386647e23 2024-05-03T17:55:19.109 INFO:teuthology.orchestra.run.smithi122.stdout:osd.10 smithi149 running (4m) 44s ago 4m 43.0M 1454M 19.0.0-3450-g9a047fe7 ee4d84620249 5e10b994845a 2024-05-03T17:55:19.109 INFO:teuthology.orchestra.run.smithi122.stdout:osd.11 smithi192 running (3m) 44s ago 3m 52.6M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 99d2745b2cfc 2024-05-03T17:55:19.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 -- 172.21.15.122:0/3041770787 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff9e00b2910 msgr2=0x7ff9e00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:19.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff9e00b2910 0x7ff9e00b4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000ac50 tx=0x7ff9f000a380 comp rx=0 tx=0).stop 2024-05-03T17:55:19.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 -- 172.21.15.122:0/3041770787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9f41068c0 msgr2=0x7ff9f419bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:19.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9f41068c0 0x7ff9f419bc60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400d980 tx=0x7ff9e400dc90 comp rx=0 tx=0).stop 2024-05-03T17:55:19.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 -- 172.21.15.122:0/3041770787 shutdown_connections 2024-05-03T17:55:19.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ff9e00b2910 0x7ff9e00b4dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:19.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9f41091f0 0x7ff9f41a2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:19.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f4103f90 0x7ff9f419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:19.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 --2- 172.21.15.122:0/3041770787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9f41068c0 0x7ff9f419bc60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:19.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 -- 172.21.15.122:0/3041770787 >> 172.21.15.122:0/3041770787 conn(0x7ff9f40fd9c0 msgr2=0x7ff9f4103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:19.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 -- 172.21.15.122:0/3041770787 shutdown_connections 2024-05-03T17:55:19.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:19.114+0000 7ff9fc226700 1 -- 172.21.15.122:0/3041770787 wait complete. 2024-05-03T17:55:19.862 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-03T17:55:20.099 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:20.471 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:20 smithi122 ceph-mon[29744]: pgmap v369: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:20.471 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:20 smithi122 ceph-mon[29744]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:20.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:20 smithi149 ceph-mon[33409]: pgmap v369: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:20.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:20 smithi149 ceph-mon[33409]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:20.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:20 smithi192 ceph-mon[33753]: pgmap v369: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:20.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:20 smithi192 ceph-mon[33753]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:21.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.906+0000 7f233fdaf700 1 -- 172.21.15.122:0/1702973837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2338102400 msgr2=0x7f2338102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:21.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.906+0000 7f233fdaf700 1 --2- 172.21.15.122:0/1702973837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2338102400 0x7f2338102820 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f23340099c0 tx=0x7f2334009cd0 comp rx=0 tx=0).stop 2024-05-03T17:55:21.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.906+0000 7f233fdaf700 1 -- 172.21.15.122:0/1702973837 shutdown_connections 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.906+0000 7f233fdaf700 1 --2- 172.21.15.122:0/1702973837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2338103fa0 0x7f233810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.906+0000 7f233fdaf700 1 --2- 172.21.15.122:0/1702973837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2338102400 0x7f2338102820 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.906+0000 7f233fdaf700 1 --2- 172.21.15.122:0/1702973837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2338103600 0x7f2338103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.906+0000 7f233fdaf700 1 -- 172.21.15.122:0/1702973837 >> 172.21.15.122:0/1702973837 conn(0x7f23380fd9a0 msgr2=0x7f23380ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.906+0000 7f233fdaf700 1 -- 172.21.15.122:0/1702973837 shutdown_connections 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.906+0000 7f233fdaf700 1 -- 172.21.15.122:0/1702973837 wait complete. 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233fdaf700 1 Processor -- start 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233fdaf700 1 -- start start 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233fdaf700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2338102400 0x7f233806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233fdaf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2338103600 0x7f233819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233fdaf700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2338103fa0 0x7f23381a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:21.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233fdaf700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f233810def0 con 0x7f2338103fa0 2024-05-03T17:55:21.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233fdaf700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f233810dd70 con 0x7f2338102400 2024-05-03T17:55:21.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233fdaf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f233810e070 con 0x7f2338103600 2024-05-03T17:55:21.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233e34c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2338103fa0 0x7f23381a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:21.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233e34c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2338103fa0 0x7f23381a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57914/0 (socket says 172.21.15.122:57914) 2024-05-03T17:55:21.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233e34c700 1 -- 172.21.15.122:0/1987260025 learned_addr learned my addr 172.21.15.122:0/1987260025 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:21.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233db4b700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2338102400 0x7f233806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:21.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233d34a700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2338103600 0x7f233819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233e34c700 1 -- 172.21.15.122:0/1987260025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2338103600 msgr2=0x7f233819e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233e34c700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2338103600 0x7f233819e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233e34c700 1 -- 172.21.15.122:0/1987260025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2338102400 msgr2=0x7f233806bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233e34c700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2338102400 0x7f233806bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.907+0000 7f233e34c700 1 -- 172.21.15.122:0/1987260025 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23381a1330 con 0x7f2338103fa0 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.908+0000 7f233e34c700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2338103fa0 0x7f23381a0a60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2330007d70 tx=0x7f233000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.908+0000 7f232affd700 1 -- 172.21.15.122:0/1987260025 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330016020 con 0x7f2338103fa0 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.908+0000 7f232affd700 1 -- 172.21.15.122:0/1987260025 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f233000ad40 con 0x7f2338103fa0 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.908+0000 7f233fdaf700 1 -- 172.21.15.122:0/1987260025 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23381a1620 con 0x7f2338103fa0 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.908+0000 7f232affd700 1 -- 172.21.15.122:0/1987260025 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23300115e0 con 0x7f2338103fa0 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.908+0000 7f233fdaf700 1 -- 172.21.15.122:0/1987260025 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23381a1b10 con 0x7f2338103fa0 2024-05-03T17:55:21.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.909+0000 7f233fdaf700 1 -- 172.21.15.122:0/1987260025 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233806a190 con 0x7f2338103fa0 2024-05-03T17:55:21.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.910+0000 7f232affd700 1 -- 172.21.15.122:0/1987260025 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2330019020 con 0x7f2338103fa0 2024-05-03T17:55:21.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.911+0000 7f232affd700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f23240b2910 0x7f23240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:21.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.911+0000 7f233db4b700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f23240b2910 0x7f23240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:21.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.911+0000 7f232affd700 1 -- 172.21.15.122:0/1987260025 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f23300e2510 con 0x7f2338103fa0 2024-05-03T17:55:21.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.911+0000 7f233db4b700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f23240b2910 0x7f23240b4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2334006010 tx=0x7f2334000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:21.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:21.914+0000 7f232affd700 1 -- 172.21.15.122:0/1987260025 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2330020020 con 0x7f2338103fa0 2024-05-03T17:55:22.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.130+0000 7f233fdaf700 1 -- 172.21.15.122:0/1987260025 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f23380638e0 con 0x7f23240b2910 2024-05-03T17:55:22.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.136+0000 7f232affd700 1 -- 172.21.15.122:0/1987260025 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f23380638e0 con 0x7f23240b2910 2024-05-03T17:55:22.131 INFO:teuthology.orchestra.run.smithi122.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-03T17:55:22.131 INFO:teuthology.orchestra.run.smithi122.stdout:mds.all 12/12 47s ago 3m smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-05-03T17:55:22.131 INFO:teuthology.orchestra.run.smithi122.stdout:mgr 3/3 47s ago 8m smithi122=x;smithi149=y;smithi192=z;count:3 2024-05-03T17:55:22.132 INFO:teuthology.orchestra.run.smithi122.stdout:mon 3/3 47s ago 9m smithi122:172.21.15.122=a;smithi149:172.21.15.149=b;smithi192:172.21.15.192=c;count:3 2024-05-03T17:55:22.132 INFO:teuthology.orchestra.run.smithi122.stdout:osd 12 47s ago - 2024-05-03T17:55:22.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 -- 172.21.15.122:0/1987260025 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f23240b2910 msgr2=0x7f23240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:22.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f23240b2910 0x7f23240b4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2334006010 tx=0x7f2334000bc0 comp rx=0 tx=0).stop 2024-05-03T17:55:22.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 -- 172.21.15.122:0/1987260025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2338103fa0 msgr2=0x7f23381a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:22.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2338103fa0 0x7f23381a0a60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2330007d70 tx=0x7f233000a340 comp rx=0 tx=0).stop 2024-05-03T17:55:22.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 -- 172.21.15.122:0/1987260025 shutdown_connections 2024-05-03T17:55:22.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f23240b2910 0x7f23240b4dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:22.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2338103fa0 0x7f23381a0a60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:22.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2338102400 0x7f233806bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:22.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 --2- 172.21.15.122:0/1987260025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2338103600 0x7f233819e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:22.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 -- 172.21.15.122:0/1987260025 >> 172.21.15.122:0/1987260025 conn(0x7f23380fd9a0 msgr2=0x7f2338106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:22.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 -- 172.21.15.122:0/1987260025 shutdown_connections 2024-05-03T17:55:22.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:22.139+0000 7f233fdaf700 1 -- 172.21.15.122:0/1987260025 wait complete. 2024-05-03T17:55:22.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:22 smithi122 ceph-mon[29744]: pgmap v370: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:22.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:22 smithi149 ceph-mon[33409]: pgmap v370: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:22.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:22 smithi192 ceph-mon[33753]: pgmap v370: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:22.956 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-03T17:55:23.193 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:23.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:23 smithi122 ceph-mon[29744]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:23.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:23 smithi149 ceph-mon[33409]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:23.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:23 smithi192 ceph-mon[33753]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:24.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:24 smithi122 ceph-mon[29744]: pgmap v371: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:24.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:24 smithi149 ceph-mon[33409]: pgmap v371: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:24.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:24 smithi192 ceph-mon[33753]: pgmap v371: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:24.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1746178610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a8103500 msgr2=0x7fc4a8103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:24.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 --2- 172.21.15.122:0/1746178610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a8103500 0x7fc4a8103920 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc4980099c0 tx=0x7fc498009cd0 comp rx=0 tx=0).stop 2024-05-03T17:55:24.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1746178610 shutdown_connections 2024-05-03T17:55:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 --2- 172.21.15.122:0/1746178610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a81050a0 0x7fc4a810b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 --2- 172.21.15.122:0/1746178610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a8103500 0x7fc4a8103920 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 --2- 172.21.15.122:0/1746178610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4a8104700 0x7fc4a8104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1746178610 >> 172.21.15.122:0/1746178610 conn(0x7fc4a80feaa0 msgr2=0x7fc4a8100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1746178610 shutdown_connections 2024-05-03T17:55:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1746178610 wait complete. 2024-05-03T17:55:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 Processor -- start 2024-05-03T17:55:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 -- start start 2024-05-03T17:55:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a8103500 0x7fc4a8077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.993+0000 7fc4ad93e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a8104700 0x7fc4a8075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4ad93e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4a81050a0 0x7fc4a8075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4ad93e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a810ef90 con 0x7fc4a8103500 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4ad93e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a810ee10 con 0x7fc4a8104700 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4ad93e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a810f110 con 0x7fc4a81050a0 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4a6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a8103500 0x7fc4a8077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4a6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a8103500 0x7fc4a8077240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57924/0 (socket says 172.21.15.122:57924) 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4a6ffd700 1 -- 172.21.15.122:0/1472085737 learned_addr learned my addr 172.21.15.122:0/1472085737 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4a77fe700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4a81050a0 0x7fc4a8075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc49ffff700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a8104700 0x7fc4a8075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4a6ffd700 1 -- 172.21.15.122:0/1472085737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4a81050a0 msgr2=0x7fc4a8075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4a6ffd700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4a81050a0 0x7fc4a8075dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4a6ffd700 1 -- 172.21.15.122:0/1472085737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a8104700 msgr2=0x7fc4a8075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4a6ffd700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a8104700 0x7fc4a8075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:24.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4a6ffd700 1 -- 172.21.15.122:0/1472085737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a8076680 con 0x7fc4a8103500 2024-05-03T17:55:24.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4a6ffd700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a8103500 0x7fc4a8077240 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc498009f50 tx=0x7fc498005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:24.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4ac93c700 1 -- 172.21.15.122:0/1472085737 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498027070 con 0x7fc4a8103500 2024-05-03T17:55:24.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4ac93c700 1 -- 172.21.15.122:0/1472085737 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc498022470 con 0x7fc4a8103500 2024-05-03T17:55:24.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4ac93c700 1 -- 172.21.15.122:0/1472085737 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49800f6c0 con 0x7fc4a8103500 2024-05-03T17:55:24.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1472085737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a81a3760 con 0x7fc4a8103500 2024-05-03T17:55:24.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.994+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1472085737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a81a3bc0 con 0x7fc4a8103500 2024-05-03T17:55:24.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.995+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1472085737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a802e510 con 0x7fc4a8103500 2024-05-03T17:55:24.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.997+0000 7fc4ac93c700 1 -- 172.21.15.122:0/1472085737 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc49802a070 con 0x7fc4a8103500 2024-05-03T17:55:24.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.998+0000 7fc4ac93c700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4940b2910 0x7fc4940b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:24.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.998+0000 7fc49ffff700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4940b2910 0x7fc4940b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:24.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.998+0000 7fc4ac93c700 1 -- 172.21.15.122:0/1472085737 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fc4980e34e0 con 0x7fc4a8103500 2024-05-03T17:55:24.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:24.998+0000 7fc49ffff700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4940b2910 0x7fc4940b4dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc490005ea0 tx=0x7fc490005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:24.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.002+0000 7fc4ac93c700 1 -- 172.21.15.122:0/1472085737 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc498031080 con 0x7fc4a8103500 2024-05-03T17:55:25.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.211+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1472085737 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc4a80638e0 con 0x7fc4940b2910 2024-05-03T17:55:25.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.212+0000 7fc4ac93c700 1 -- 172.21.15.122:0/1472085737 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc4a80638e0 con 0x7fc4940b2910 2024-05-03T17:55:25.207 INFO:teuthology.orchestra.run.smithi122.stdout:HOST ADDR LABELS STATUS 2024-05-03T17:55:25.207 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 172.21.15.122 2024-05-03T17:55:25.207 INFO:teuthology.orchestra.run.smithi122.stdout:smithi149 172.21.15.149 2024-05-03T17:55:25.207 INFO:teuthology.orchestra.run.smithi122.stdout:smithi192 172.21.15.192 2024-05-03T17:55:25.207 INFO:teuthology.orchestra.run.smithi122.stdout:3 hosts in cluster 2024-05-03T17:55:25.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.214+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1472085737 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4940b2910 msgr2=0x7fc4940b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:25.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.214+0000 7fc4ad93e700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4940b2910 0x7fc4940b4dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc490005ea0 tx=0x7fc490005e30 comp rx=0 tx=0).stop 2024-05-03T17:55:25.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.214+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1472085737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a8103500 msgr2=0x7fc4a8077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:25.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.214+0000 7fc4ad93e700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a8103500 0x7fc4a8077240 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc498009f50 tx=0x7fc498005e80 comp rx=0 tx=0).stop 2024-05-03T17:55:25.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.214+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1472085737 shutdown_connections 2024-05-03T17:55:25.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.214+0000 7fc4ad93e700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc4940b2910 0x7fc4940b4dd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:25.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.214+0000 7fc4ad93e700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4a81050a0 0x7fc4a8075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:25.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.214+0000 7fc4ad93e700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a8103500 0x7fc4a8077240 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:25.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.214+0000 7fc4ad93e700 1 --2- 172.21.15.122:0/1472085737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a8104700 0x7fc4a8075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:25.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.214+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1472085737 >> 172.21.15.122:0/1472085737 conn(0x7fc4a80feaa0 msgr2=0x7fc4a8107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:25.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.215+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1472085737 shutdown_connections 2024-05-03T17:55:25.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:25.215+0000 7fc4ad93e700 1 -- 172.21.15.122:0/1472085737 wait complete. 2024-05-03T17:55:25.960 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-03T17:55:26.196 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:26.645 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:26 smithi122 ceph-mon[29744]: pgmap v372: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:26.645 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:26 smithi122 ceph-mon[29744]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:26.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:26 smithi149 ceph-mon[33409]: pgmap v372: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:26.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:26 smithi149 ceph-mon[33409]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:26.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:26 smithi192 ceph-mon[33753]: pgmap v372: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:26.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:26 smithi192 ceph-mon[33753]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.993+0000 7fe1bcfba700 1 -- 172.21.15.122:0/3375691252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1b81091e0 msgr2=0x7fe1b810b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.993+0000 7fe1bcfba700 1 --2- 172.21.15.122:0/3375691252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1b81091e0 0x7fe1b810b650 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009990 tx=0x7fe1a8009ca0 comp rx=0 tx=0).stop 2024-05-03T17:55:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.993+0000 7fe1bcfba700 1 -- 172.21.15.122:0/3375691252 shutdown_connections 2024-05-03T17:55:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.993+0000 7fe1bcfba700 1 --2- 172.21.15.122:0/3375691252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1b81091e0 0x7fe1b810b650 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.993+0000 7fe1bcfba700 1 --2- 172.21.15.122:0/3375691252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1b8103f80 0x7fe1b8106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.993+0000 7fe1bcfba700 1 --2- 172.21.15.122:0/3375691252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b81068b0 0x7fe1b8108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.993+0000 7fe1bcfba700 1 -- 172.21.15.122:0/3375691252 >> 172.21.15.122:0/3375691252 conn(0x7fe1b80fd990 msgr2=0x7fe1b80ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.993+0000 7fe1bcfba700 1 -- 172.21.15.122:0/3375691252 shutdown_connections 2024-05-03T17:55:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.993+0000 7fe1bcfba700 1 -- 172.21.15.122:0/3375691252 wait complete. 2024-05-03T17:55:27.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.994+0000 7fe1bcfba700 1 Processor -- start 2024-05-03T17:55:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.994+0000 7fe1bcfba700 1 -- start start 2024-05-03T17:55:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.994+0000 7fe1bcfba700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1b8103f80 0x7fe1b81995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.994+0000 7fe1bcfba700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1b81068b0 0x7fe1b8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.994+0000 7fe1bcfba700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b81091e0 0x7fe1b81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.994+0000 7fe1bcfba700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b810efd0 con 0x7fe1b8103f80 2024-05-03T17:55:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.994+0000 7fe1bcfba700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b810ee50 con 0x7fe1b81091e0 2024-05-03T17:55:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.994+0000 7fe1bcfba700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b810f150 con 0x7fe1b81068b0 2024-05-03T17:55:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.994+0000 7fe1b659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1b8103f80 0x7fe1b81995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.994+0000 7fe1b659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1b8103f80 0x7fe1b81995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57948/0 (socket says 172.21.15.122:57948) 2024-05-03T17:55:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.994+0000 7fe1b659c700 1 -- 172.21.15.122:0/115046132 learned_addr learned my addr 172.21.15.122:0/115046132 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1b6d9d700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b81091e0 0x7fe1b81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1b5d9b700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1b81068b0 0x7fe1b8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1b659c700 1 -- 172.21.15.122:0/115046132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1b81068b0 msgr2=0x7fe1b8199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1b659c700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1b81068b0 0x7fe1b8199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1b659c700 1 -- 172.21.15.122:0/115046132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b81091e0 msgr2=0x7fe1b81a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1b659c700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b81091e0 0x7fe1b81a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1b659c700 1 -- 172.21.15.122:0/115046132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b81a1240 con 0x7fe1b8103f80 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1b659c700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1b8103f80 0x7fe1b81995e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000ba80 tx=0x7fe1b000bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1a77fe700 1 -- 172.21.15.122:0/115046132 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0019070 con 0x7fe1b8103f80 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1a77fe700 1 -- 172.21.15.122:0/115046132 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b0014440 con 0x7fe1b8103f80 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1bcfba700 1 -- 172.21.15.122:0/115046132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b81a1530 con 0x7fe1b8103f80 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1a77fe700 1 -- 172.21.15.122:0/115046132 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000ce60 con 0x7fe1b8103f80 2024-05-03T17:55:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.995+0000 7fe1bcfba700 1 -- 172.21.15.122:0/115046132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b81a1af0 con 0x7fe1b8103f80 2024-05-03T17:55:27.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.996+0000 7fe1bcfba700 1 -- 172.21.15.122:0/115046132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe198005360 con 0x7fe1b8103f80 2024-05-03T17:55:27.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.997+0000 7fe1a77fe700 1 -- 172.21.15.122:0/115046132 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe1b001c070 con 0x7fe1b8103f80 2024-05-03T17:55:27.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.998+0000 7fe1a77fe700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe1a00b2910 0x7fe1a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:27.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.998+0000 7fe1b5d9b700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe1a00b2910 0x7fe1a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:27.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.998+0000 7fe1a77fe700 1 -- 172.21.15.122:0/115046132 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fe1b00e1440 con 0x7fe1b8103f80 2024-05-03T17:55:27.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:27.999+0000 7fe1b5d9b700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe1a00b2910 0x7fe1a00b4dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac005ea0 tx=0x7fe1ac005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:27.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.003+0000 7fe1a77fe700 1 -- 172.21.15.122:0/115046132 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe1b00a92a0 con 0x7fe1b8103f80 2024-05-03T17:55:28.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.214+0000 7fe1bcfba700 1 -- 172.21.15.122:0/115046132 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe198000bf0 con 0x7fe1a00b2910 2024-05-03T17:55:28.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.216+0000 7fe1a77fe700 1 -- 172.21.15.122:0/115046132 <== mgr.14152 v2:172.21.15.122:6800/1089873263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe198000bf0 con 0x7fe1a00b2910 2024-05-03T17:55:28.211 INFO:teuthology.orchestra.run.smithi122.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-03T17:55:28.211 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010U400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-03T17:55:28.211 INFO:teuthology.orchestra.run.smithi122.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-03T17:55:28.211 INFO:teuthology.orchestra.run.smithi122.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-03T17:55:28.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 -- 172.21.15.122:0/115046132 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe1a00b2910 msgr2=0x7fe1a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:28.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe1a00b2910 0x7fe1a00b4dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac005ea0 tx=0x7fe1ac005e30 comp rx=0 tx=0).stop 2024-05-03T17:55:28.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 -- 172.21.15.122:0/115046132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1b8103f80 msgr2=0x7fe1b81995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:28.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1b8103f80 0x7fe1b81995e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000ba80 tx=0x7fe1b000bd90 comp rx=0 tx=0).stop 2024-05-03T17:55:28.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 -- 172.21.15.122:0/115046132 shutdown_connections 2024-05-03T17:55:28.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fe1a00b2910 0x7fe1a00b4dd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:28.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b81091e0 0x7fe1b81a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:28.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1b8103f80 0x7fe1b81995e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:28.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 --2- 172.21.15.122:0/115046132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1b81068b0 0x7fe1b8199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:28.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 -- 172.21.15.122:0/115046132 >> 172.21.15.122:0/115046132 conn(0x7fe1b80fd990 msgr2=0x7fe1b81048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:28.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 -- 172.21.15.122:0/115046132 shutdown_connections 2024-05-03T17:55:28.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:28.218+0000 7fe1bcfba700 1 -- 172.21.15.122:0/115046132 wait complete. 2024-05-03T17:55:28.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:28 smithi122 ceph-mon[29744]: pgmap v373: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:28.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:28 smithi149 ceph-mon[33409]: pgmap v373: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:28.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:28 smithi192 ceph-mon[33753]: pgmap v373: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:28.931 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-03T17:55:28.940 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-05-03T17:55:28.940 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-03T17:55:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:29.622 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:29 smithi122 ceph-mon[29744]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:29.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:29 smithi149 ceph-mon[33409]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:29.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:29 smithi192 ceph-mon[33753]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:30.641 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:30 smithi122 ceph-mon[29744]: pgmap v374: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:30.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:30 smithi149 ceph-mon[33409]: pgmap v374: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:30.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:30 smithi192 ceph-mon[33753]: pgmap v374: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:30.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.906+0000 7f17bcf2b700 1 -- 172.21.15.122:0/723045681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17b8103620 msgr2=0x7f17b8103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.906+0000 7f17bcf2b700 1 --2- 172.21.15.122:0/723045681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17b8103620 0x7f17b8103a80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f17a00099c0 tx=0x7f17a0009cd0 comp rx=0 tx=0).stop 2024-05-03T17:55:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.906+0000 7f17bcf2b700 1 -- 172.21.15.122:0/723045681 shutdown_connections 2024-05-03T17:55:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.906+0000 7f17bcf2b700 1 --2- 172.21.15.122:0/723045681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b8103fc0 0x7f17b810a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.906+0000 7f17bcf2b700 1 --2- 172.21.15.122:0/723045681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b8102420 0x7f17b8102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.906+0000 7f17bcf2b700 1 --2- 172.21.15.122:0/723045681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17b8103620 0x7f17b8103a80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.906+0000 7f17bcf2b700 1 -- 172.21.15.122:0/723045681 >> 172.21.15.122:0/723045681 conn(0x7f17b80fd9c0 msgr2=0x7f17b80ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.906+0000 7f17bcf2b700 1 -- 172.21.15.122:0/723045681 shutdown_connections 2024-05-03T17:55:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.906+0000 7f17bcf2b700 1 -- 172.21.15.122:0/723045681 wait complete. 2024-05-03T17:55:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.906+0000 7f17bcf2b700 1 Processor -- start 2024-05-03T17:55:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17bcf2b700 1 -- start start 2024-05-03T17:55:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17bcf2b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b8102420 0x7f17b81063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17bcf2b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b8103620 0x7f17b8104a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17bcf2b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17b8103fc0 0x7f17b8104f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17bcf2b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17b810dea0 con 0x7f17b8103fc0 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17bcf2b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f17b810dd20 con 0x7f17b8102420 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17bcf2b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f17b810e020 con 0x7f17b8103620 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17b6d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17b8103fc0 0x7f17b8104f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17b6d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17b8103fc0 0x7f17b8104f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44966/0 (socket says 172.21.15.122:44966) 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17b6d9d700 1 -- 172.21.15.122:0/376363058 learned_addr learned my addr 172.21.15.122:0/376363058 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17add9b700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b8103620 0x7f17b8104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17b659c700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b8102420 0x7f17b81063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17b6d9d700 1 -- 172.21.15.122:0/376363058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b8103620 msgr2=0x7f17b8104a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17b6d9d700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b8103620 0x7f17b8104a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17b6d9d700 1 -- 172.21.15.122:0/376363058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b8102420 msgr2=0x7f17b81063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17b6d9d700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b8102420 0x7f17b81063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:30.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17b6d9d700 1 -- 172.21.15.122:0/376363058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b81057f0 con 0x7f17b8103fc0 2024-05-03T17:55:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17add9b700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b8103620 0x7f17b8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.907+0000 7f17b6d9d700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17b8103fc0 0x7f17b8104f40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f17b000ada0 tx=0x7f17b000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.908+0000 7f17affff700 1 -- 172.21.15.122:0/376363058 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0017020 con 0x7f17b8103fc0 2024-05-03T17:55:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.908+0000 7f17bcf2b700 1 -- 172.21.15.122:0/376363058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b81a37d0 con 0x7f17b8103fc0 2024-05-03T17:55:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.908+0000 7f17affff700 1 -- 172.21.15.122:0/376363058 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17b000ce30 con 0x7f17b8103fc0 2024-05-03T17:55:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.908+0000 7f17affff700 1 -- 172.21.15.122:0/376363058 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b00054a0 con 0x7f17b8103fc0 2024-05-03T17:55:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.908+0000 7f17bcf2b700 1 -- 172.21.15.122:0/376363058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b81a3c90 con 0x7f17b8103fc0 2024-05-03T17:55:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.909+0000 7f17bcf2b700 1 -- 172.21.15.122:0/376363058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b806a190 con 0x7f17b8103fc0 2024-05-03T17:55:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.910+0000 7f17affff700 1 -- 172.21.15.122:0/376363058 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f17b000c980 con 0x7f17b8103fc0 2024-05-03T17:55:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.911+0000 7f17affff700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f17a40b2910 0x7f17a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.911+0000 7f17b659c700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f17a40b2910 0x7f17a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.911+0000 7f17affff700 1 -- 172.21.15.122:0/376363058 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f17b00e2110 con 0x7f17b8103fc0 2024-05-03T17:55:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.911+0000 7f17b659c700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f17a40b2910 0x7f17a40b4dd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f17a8005fd0 tx=0x7f17a8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:30.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:30.915+0000 7f17affff700 1 -- 172.21.15.122:0/376363058 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f17b0021020 con 0x7f17b8103fc0 2024-05-03T17:55:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.162+0000 7f17bcf2b700 1 -- 172.21.15.122:0/376363058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f17b81a4010 con 0x7f17b8103fc0 2024-05-03T17:55:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.163+0000 7f17affff700 1 -- 172.21.15.122:0/376363058 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3064 (secure 0 0 0) 0x7f17b00a9de0 con 0x7f17b8103fc0 2024-05-03T17:55:31.158 INFO:teuthology.orchestra.run.smithi122.stdout:e24 2024-05-03T17:55:31.158 INFO:teuthology.orchestra.run.smithi122.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T17:55:31.158 INFO:teuthology.orchestra.run.smithi122.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T17:55:31.158 INFO:teuthology.orchestra.run.smithi122.stdout:legacy client fscid: 1 2024-05-03T17:55:31.158 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:55:31.158 INFO:teuthology.orchestra.run.smithi122.stdout:Filesystem 'cephfs' (1) 2024-05-03T17:55:31.158 INFO:teuthology.orchestra.run.smithi122.stdout:fs_name cephfs 2024-05-03T17:55:31.158 INFO:teuthology.orchestra.run.smithi122.stdout:epoch 24 2024-05-03T17:55:31.158 INFO:teuthology.orchestra.run.smithi122.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-03T17:55:31.158 INFO:teuthology.orchestra.run.smithi122.stdout:created 2024-05-03T17:52:37.616728+0000 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:modified 2024-05-03T17:52:56.867078+0000 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:tableserver 0 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:root 0 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:session_timeout 300 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:session_autoclose 300 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:max_file_size 1099511627776 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:max_xattr_size 65536 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:required_client_features {} 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:last_failure 0 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:last_failure_osd_epoch 0 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:max_mds 3 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:in 0,1,2 2024-05-03T17:55:31.159 INFO:teuthology.orchestra.run.smithi122.stdout:up {0=24481,1=24463,2=24457} 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:failed 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:damaged 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:stopped 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:data_pools [3] 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:metadata_pool 2 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:inline_data disabled 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:balancer 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:bal_rank_mask -1 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:standby_count_wanted 0 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:qdb_cluster leader: 24481 members: 24481,24457,24463 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.i{0:24481} state up:active seq 8 addr [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b{0:24443} state up:standby-replay seq 1 addr [v2:172.21.15.149:6838/1529632960,v1:172.21.15.149:6839/1529632960] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.c{1:24463} state up:active seq 13 addr [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.l{1:24421} state up:standby-replay seq 1 addr [v2:172.21.15.192:6832/899389349,v1:172.21.15.192:6833/899389349] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.160 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.f{2:24457} state up:active seq 14 addr [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.161 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.h{2:24410} state up:standby-replay seq 1 addr [v2:172.21.15.149:6836/2599524712,v1:172.21.15.149:6837/2599524712] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.161 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:55:31.161 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:55:31.161 INFO:teuthology.orchestra.run.smithi122.stdout:Standby daemons: 2024-05-03T17:55:31.161 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:55:31.161 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.j{-1:14559} state up:standby seq 1 addr [v2:172.21.15.122:6834/2168395467,v1:172.21.15.122:6835/2168395467] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.161 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.d{-1:14571} state up:standby seq 1 addr [v2:172.21.15.122:6836/3093239187,v1:172.21.15.122:6837/3093239187] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.161 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.g{-1:14619} state up:standby seq 1 addr [v2:172.21.15.122:6838/555236113,v1:172.21.15.122:6839/555236113] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.161 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a{-1:14631} state up:standby seq 1 addr [v2:172.21.15.122:6840/1783292878,v1:172.21.15.122:6841/1783292878] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.161 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.k{-1:24371} state up:standby seq 1 addr [v2:172.21.15.149:6832/3370166837,v1:172.21.15.149:6833/3370166837] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.161 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.e{-1:24389} state up:standby seq 1 addr [v2:172.21.15.149:6834/1470504763,v1:172.21.15.149:6835/1470504763] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T17:55:31.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 -- 172.21.15.122:0/376363058 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f17a40b2910 msgr2=0x7f17a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:31.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f17a40b2910 0x7f17a40b4dd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f17a8005fd0 tx=0x7f17a8005e50 comp rx=0 tx=0).stop 2024-05-03T17:55:31.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 -- 172.21.15.122:0/376363058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17b8103fc0 msgr2=0x7f17b8104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:31.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17b8103fc0 0x7f17b8104f40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f17b000ada0 tx=0x7f17b000c330 comp rx=0 tx=0).stop 2024-05-03T17:55:31.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 -- 172.21.15.122:0/376363058 shutdown_connections 2024-05-03T17:55:31.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f17a40b2910 0x7f17a40b4dd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:31.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17b8103fc0 0x7f17b8104f40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:31.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b8102420 0x7f17b81063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:31.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 --2- 172.21.15.122:0/376363058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b8103620 0x7f17b8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:31.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 -- 172.21.15.122:0/376363058 >> 172.21.15.122:0/376363058 conn(0x7f17b80fd9c0 msgr2=0x7f17b80ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:31.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 -- 172.21.15.122:0/376363058 shutdown_connections 2024-05-03T17:55:31.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:31.166+0000 7f17bcf2b700 1 -- 172.21.15.122:0/376363058 wait complete. 2024-05-03T17:55:31.163 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 24 2024-05-03T17:55:31.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:31 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/376363058' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T17:55:31.742 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-03T17:55:31.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:31 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/376363058' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T17:55:31.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:31 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/376363058' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T17:55:31.981 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:32.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:32 smithi122 ceph-mon[29744]: pgmap v375: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:32.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:32 smithi192 ceph-mon[33753]: pgmap v375: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:32 smithi149 ceph-mon[33409]: pgmap v375: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:33.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.730+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/3815704374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f81023f0 msgr2=0x7fc7f8102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:33.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.730+0000 7fc7ff7b4700 1 --2- 172.21.15.122:0/3815704374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f81023f0 0x7fc7f8102810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400b0d0 tx=0x7fc7f400b3e0 comp rx=0 tx=0).stop 2024-05-03T17:55:33.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.730+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/3815704374 shutdown_connections 2024-05-03T17:55:33.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.730+0000 7fc7ff7b4700 1 --2- 172.21.15.122:0/3815704374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f8103f90 0x7fc7f810a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:33.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.730+0000 7fc7ff7b4700 1 --2- 172.21.15.122:0/3815704374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f81023f0 0x7fc7f8102810 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:33.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.730+0000 7fc7ff7b4700 1 --2- 172.21.15.122:0/3815704374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f81035f0 0x7fc7f8103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:33.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.730+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/3815704374 >> 172.21.15.122:0/3815704374 conn(0x7fc7f80fd990 msgr2=0x7fc7f80ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.730+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/3815704374 shutdown_connections 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.730+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/3815704374 wait complete. 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.730+0000 7fc7ff7b4700 1 Processor -- start 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7ff7b4700 1 -- start start 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7ff7b4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f81023f0 0x7fc7f8106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7ff7b4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f81035f0 0x7fc7f81049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7ff7b4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f8103f90 0x7fc7f8104f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7ff7b4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f810de60 con 0x7fc7f8103f90 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7ff7b4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f810dce0 con 0x7fc7f81023f0 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7ff7b4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f810dfe0 con 0x7fc7f81035f0 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fdd51700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f8103f90 0x7fc7f8104f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fdd51700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f8103f90 0x7fc7f8104f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44980/0 (socket says 172.21.15.122:44980) 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fdd51700 1 -- 172.21.15.122:0/1777219704 learned_addr learned my addr 172.21.15.122:0/1777219704 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fd550700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f81023f0 0x7fc7f8106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fcd4f700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f81035f0 0x7fc7f81049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fdd51700 1 -- 172.21.15.122:0/1777219704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f81035f0 msgr2=0x7fc7f81049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fdd51700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f81035f0 0x7fc7f81049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fdd51700 1 -- 172.21.15.122:0/1777219704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f81023f0 msgr2=0x7fc7f8106380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fdd51700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f81023f0 0x7fc7f8106380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fdd51700 1 -- 172.21.15.122:0/1777219704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f81057c0 con 0x7fc7f8103f90 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fd550700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f81023f0 0x7fc7f8106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7fdd51700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f8103f90 0x7fc7f8104f10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000d980 tx=0x7fc7f000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.731+0000 7fc7ee7fc700 1 -- 172.21.15.122:0/1777219704 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0017070 con 0x7fc7f8103f90 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.732+0000 7fc7ee7fc700 1 -- 172.21.15.122:0/1777219704 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f0009df0 con 0x7fc7f8103f90 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.732+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/1777219704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f81a37d0 con 0x7fc7f8103f90 2024-05-03T17:55:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.732+0000 7fc7ee7fc700 1 -- 172.21.15.122:0/1777219704 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f00053a0 con 0x7fc7f8103f90 2024-05-03T17:55:33.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.732+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/1777219704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f81a3d10 con 0x7fc7f8103f90 2024-05-03T17:55:33.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.732+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/1777219704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f806a190 con 0x7fc7f8103f90 2024-05-03T17:55:33.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.734+0000 7fc7ee7fc700 1 -- 172.21.15.122:0/1777219704 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc7f0004750 con 0x7fc7f8103f90 2024-05-03T17:55:33.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.734+0000 7fc7ee7fc700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc7e40b2990 0x7fc7e40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:33.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.735+0000 7fc7fd550700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc7e40b2990 0x7fc7e40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:33.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.735+0000 7fc7ee7fc700 1 -- 172.21.15.122:0/1777219704 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fc7f00e2600 con 0x7fc7f8103f90 2024-05-03T17:55:33.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.735+0000 7fc7fd550700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc7e40b2990 0x7fc7e40b4e50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400bf60 tx=0x7fc7f400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:33.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.738+0000 7fc7ee7fc700 1 -- 172.21.15.122:0/1777219704 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc7f0020020 con 0x7fc7f8103f90 2024-05-03T17:55:33.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.941+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/1777219704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fc7f81a4130 con 0x7fc7f8103f90 2024-05-03T17:55:33.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.942+0000 7fc7ee7fc700 1 -- 172.21.15.122:0/1777219704 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v79) ==== 56+0+5255 (secure 0 0 0) 0x7fc7f00aa340 con 0x7fc7f8103f90 2024-05-03T17:55:33.937 INFO:teuthology.orchestra.run.smithi122.stdout:epoch 79 2024-05-03T17:55:33.937 INFO:teuthology.orchestra.run.smithi122.stdout:fsid a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T17:55:33.937 INFO:teuthology.orchestra.run.smithi122.stdout:created 2024-05-03T17:44:17.415571+0000 2024-05-03T17:55:33.937 INFO:teuthology.orchestra.run.smithi122.stdout:modified 2024-05-03T17:53:19.863832+0000 2024-05-03T17:55:33.937 INFO:teuthology.orchestra.run.smithi122.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-03T17:55:33.937 INFO:teuthology.orchestra.run.smithi122.stdout:crush_version 26 2024-05-03T17:55:33.937 INFO:teuthology.orchestra.run.smithi122.stdout:full_ratio 0.95 2024-05-03T17:55:33.938 INFO:teuthology.orchestra.run.smithi122.stdout:backfillfull_ratio 0.9 2024-05-03T17:55:33.938 INFO:teuthology.orchestra.run.smithi122.stdout:nearfull_ratio 0.85 2024-05-03T17:55:33.938 INFO:teuthology.orchestra.run.smithi122.stdout:require_min_compat_client luminous 2024-05-03T17:55:33.938 INFO:teuthology.orchestra.run.smithi122.stdout:min_compat_client luminous 2024-05-03T17:55:33.938 INFO:teuthology.orchestra.run.smithi122.stdout:require_osd_release squid 2024-05-03T17:55:33.938 INFO:teuthology.orchestra.run.smithi122.stdout:stretch_mode_enabled false 2024-05-03T17:55:33.938 INFO:teuthology.orchestra.run.smithi122.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-03T17:55:33.938 INFO:teuthology.orchestra.run.smithi122.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-03T17:55:33.938 INFO:teuthology.orchestra.run.smithi122.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-03T17:55:33.938 INFO:teuthology.orchestra.run.smithi122.stdout:max_osd 12 2024-05-03T17:55:33.938 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6802/1329364610,v1:172.21.15.122:6803/1329364610] [v2:172.21.15.122:6804/1329364610,v1:172.21.15.122:6805/1329364610] exists,up a4b08628-0a29-44aa-b8b9-ed9657f12be6 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 up in weight 1 up_from 13 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/1402463302,v1:172.21.15.149:6801/1402463302] [v2:172.21.15.149:6802/1402463302,v1:172.21.15.149:6803/1402463302] exists,up ba550787-ce76-4944-8ebd-97e19590bb3b 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 up in weight 1 up_from 18 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/3757191110,v1:172.21.15.192:6801/3757191110] [v2:172.21.15.192:6802/3757191110,v1:172.21.15.192:6803/3757191110] exists,up 35ae7c47-d710-47c5-b0ec-91fa12b5be07 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 up in weight 1 up_from 25 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6810/2359143764,v1:172.21.15.122:6811/2359143764] [v2:172.21.15.122:6812/2359143764,v1:172.21.15.122:6813/2359143764] exists,up 9973c0c1-ae73-497a-845e-664a20666df6 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 up in weight 1 up_from 31 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/3627037532,v1:172.21.15.149:6809/3627037532] [v2:172.21.15.149:6810/3627037532,v1:172.21.15.149:6811/3627037532] exists,up da71dc18-94b9-4f7d-a061-7cc7b77db282 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 up in weight 1 up_from 36 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/455779485,v1:172.21.15.192:6809/455779485] [v2:172.21.15.192:6810/455779485,v1:172.21.15.192:6811/455779485] exists,up bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 up in weight 1 up_from 42 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6818/3419421998,v1:172.21.15.122:6819/3419421998] [v2:172.21.15.122:6820/3419421998,v1:172.21.15.122:6821/3419421998] exists,up 4150bfe5-a0e4-4bae-914a-75598f017519 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 up in weight 1 up_from 47 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/139747549,v1:172.21.15.149:6817/139747549] [v2:172.21.15.149:6818/139747549,v1:172.21.15.149:6819/139747549] exists,up 71e01812-e511-44a0-a143-2afca510ae8b 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:osd.8 up in weight 1 up_from 53 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/2080816744,v1:172.21.15.192:6817/2080816744] [v2:172.21.15.192:6818/2080816744,v1:172.21.15.192:6819/2080816744] exists,up 5294516d-fac6-41d0-a635-6bc1d7eca585 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:osd.9 up in weight 1 up_from 59 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6826/1773691421,v1:172.21.15.122:6827/1773691421] [v2:172.21.15.122:6828/1773691421,v1:172.21.15.122:6829/1773691421] exists,up f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:osd.10 up in weight 1 up_from 64 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/1954186137,v1:172.21.15.149:6825/1954186137] [v2:172.21.15.149:6826/1954186137,v1:172.21.15.149:6827/1954186137] exists,up af88e50f-98f1-460d-856b-a920dd2ae514 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:osd.11 up in weight 1 up_from 69 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/2098474209,v1:172.21.15.192:6825/2098474209] [v2:172.21.15.192:6826/2098474209,v1:172.21.15.192:6827/2098474209] exists,up b54c1b4d-cdb5-42d9-979a-47f9a870b5cd 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 2.4 [2,11] 2024-05-03T17:55:33.939 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 2.1c [8,11] 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.1 [4,9] 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.3 [4,9] 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.1d [5,9] 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.31 [4,8] 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/2089907395 expires 2024-05-04T17:45:18.454406+0000 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6801/1659741627 expires 2024-05-04T17:45:18.454406+0000 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/2001860625 expires 2024-05-04T17:44:52.244049+0000 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/1888284988 expires 2024-05-04T17:44:52.244049+0000 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/256574472 expires 2024-05-04T17:44:52.244049+0000 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6801/80220221 expires 2024-05-04T17:44:52.244049+0000 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/64546794 expires 2024-05-04T17:45:18.454406+0000 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6800/1659741627 expires 2024-05-04T17:45:18.454406+0000 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6800/80220221 expires 2024-05-04T17:44:52.244049+0000 2024-05-03T17:55:33.940 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/584846864 expires 2024-05-04T17:45:18.454406+0000 2024-05-03T17:55:33.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/1777219704 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc7e40b2990 msgr2=0x7fc7e40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:33.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc7e40b2990 0x7fc7e40b4e50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400bf60 tx=0x7fc7f400b040 comp rx=0 tx=0).stop 2024-05-03T17:55:33.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/1777219704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f8103f90 msgr2=0x7fc7f8104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f8103f90 0x7fc7f8104f10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000d980 tx=0x7fc7f000dd40 comp rx=0 tx=0).stop 2024-05-03T17:55:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/1777219704 shutdown_connections 2024-05-03T17:55:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fc7e40b2990 0x7fc7e40b4e50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f8103f90 0x7fc7f8104f10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7f81023f0 0x7fc7f8106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 --2- 172.21.15.122:0/1777219704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f81035f0 0x7fc7f81049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/1777219704 >> 172.21.15.122:0/1777219704 conn(0x7fc7f80fd990 msgr2=0x7fc7f80ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/1777219704 shutdown_connections 2024-05-03T17:55:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:33.944+0000 7fc7ff7b4700 1 -- 172.21.15.122:0/1777219704 wait complete. 2024-05-03T17:55:34.686 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:34 smithi122 ceph-mon[29744]: pgmap v376: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:34.686 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:34 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1777219704' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T17:55:34.716 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-03T17:55:34.730 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:55:34.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:34 smithi149 ceph-mon[33409]: pgmap v376: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:34.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:34 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/1777219704' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T17:55:34.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:34 smithi192 ceph-mon[33753]: pgmap v376: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:34.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:34 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/1777219704' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T17:55:34.968 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:36.634 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:36 smithi122 ceph-mon[29744]: pgmap v377: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:36.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.637+0000 7fbebd30b700 1 -- 172.21.15.122:0/3128944163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb8108850 msgr2=0x7fbeb81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:36.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.637+0000 7fbebd30b700 1 --2- 172.21.15.122:0/3128944163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb8108850 0x7fbeb81103d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000b0d0 tx=0x7fbeb000b3e0 comp rx=0 tx=0).stop 2024-05-03T17:55:36.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.637+0000 7fbebd30b700 1 -- 172.21.15.122:0/3128944163 shutdown_connections 2024-05-03T17:55:36.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.637+0000 7fbebd30b700 1 --2- 172.21.15.122:0/3128944163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb8108850 0x7fbeb81103d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:36.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.637+0000 7fbebd30b700 1 --2- 172.21.15.122:0/3128944163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8103750 0x7fbeb8103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:36.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.637+0000 7fbebd30b700 1 --2- 172.21.15.122:0/3128944163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb8104100 0x7fbeb8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:36.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.637+0000 7fbebd30b700 1 -- 172.21.15.122:0/3128944163 >> 172.21.15.122:0/3128944163 conn(0x7fbeb807a600 msgr2=0x7fbeb807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:36.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.638+0000 7fbebd30b700 1 -- 172.21.15.122:0/3128944163 shutdown_connections 2024-05-03T17:55:36.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.638+0000 7fbebd30b700 1 -- 172.21.15.122:0/3128944163 wait complete. 2024-05-03T17:55:36.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.639+0000 7fbebd30b700 1 Processor -- start 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.639+0000 7fbebd30b700 1 -- start start 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.639+0000 7fbebd30b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb8103750 0x7fbeb819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.639+0000 7fbebd30b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb8104100 0x7fbeb819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.639+0000 7fbebd30b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8108850 0x7fbeb81998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.639+0000 7fbebd30b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8113640 con 0x7fbeb8103750 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.639+0000 7fbebd30b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81134c0 con 0x7fbeb8104100 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.639+0000 7fbebd30b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81137c0 con 0x7fbeb8108850 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.639+0000 7fbeb6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb8103750 0x7fbeb819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.639+0000 7fbeb6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb8103750 0x7fbeb819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44986/0 (socket says 172.21.15.122:44986) 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbeb6ffd700 1 -- 172.21.15.122:0/2891843781 learned_addr learned my addr 172.21.15.122:0/2891843781 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbeb67fc700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb8104100 0x7fbeb819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbeb77fe700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8108850 0x7fbeb81998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbeb6ffd700 1 -- 172.21.15.122:0/2891843781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8108850 msgr2=0x7fbeb81998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbeb6ffd700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8108850 0x7fbeb81998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbeb6ffd700 1 -- 172.21.15.122:0/2891843781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb8104100 msgr2=0x7fbeb819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbeb6ffd700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb8104100 0x7fbeb819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbeb6ffd700 1 -- 172.21.15.122:0/2891843781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb819a040 con 0x7fbeb8103750 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbeb67fc700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb8104100 0x7fbeb819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbeb6ffd700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb8103750 0x7fbeb819f6d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbea800d980 tx=0x7fbea800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbe9ffff700 1 -- 172.21.15.122:0/2891843781 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8017070 con 0x7fbeb8103750 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbebd30b700 1 -- 172.21.15.122:0/2891843781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb819a330 con 0x7fbeb8103750 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbe9ffff700 1 -- 172.21.15.122:0/2891843781 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea8005020 con 0x7fbeb8103750 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbe9ffff700 1 -- 172.21.15.122:0/2891843781 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8009b50 con 0x7fbeb8103750 2024-05-03T17:55:36.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.640+0000 7fbebd30b700 1 -- 172.21.15.122:0/2891843781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb81a65a0 con 0x7fbeb8103750 2024-05-03T17:55:36.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.641+0000 7fbebd30b700 1 -- 172.21.15.122:0/2891843781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb806a190 con 0x7fbeb8103750 2024-05-03T17:55:36.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.642+0000 7fbe9ffff700 1 -- 172.21.15.122:0/2891843781 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fbea80051d0 con 0x7fbeb8103750 2024-05-03T17:55:36.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.643+0000 7fbe9ffff700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbea00b2910 0x7fbea00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:36.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.643+0000 7fbeb67fc700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbea00b2910 0x7fbea00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:36.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.644+0000 7fbe9ffff700 1 -- 172.21.15.122:0/2891843781 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fbea80e26b0 con 0x7fbeb8103750 2024-05-03T17:55:36.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.644+0000 7fbeb67fc700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbea00b2910 0x7fbea00b4dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8103040 tx=0x7fbeac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:36.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.647+0000 7fbe9ffff700 1 -- 172.21.15.122:0/2891843781 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbea8020020 con 0x7fbeb8103750 2024-05-03T17:55:36.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:36 smithi149 ceph-mon[33409]: pgmap v377: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:36.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:36 smithi192 ceph-mon[33753]: pgmap v377: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:36.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.899+0000 7fbebd30b700 1 -- 172.21.15.122:0/2891843781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbeb802e510 con 0x7fbeb8103750 2024-05-03T17:55:36.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.900+0000 7fbe9ffff700 1 -- 172.21.15.122:0/2891843781 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11892 (secure 0 0 0) 0x7fbea80aa2f0 con 0x7fbeb8103750 2024-05-03T17:55:36.895 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:55:36.895 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/555236113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":555236113},{"type":"v1","addr":"172.21.15.122:6839","nonce":555236113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1783292878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1783292878},{"type":"v1","addr":"172.21.15.122:6841","nonce":1783292878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T17:52:37.616728+0000","modified":"2024-05-03T17:52:56.867078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24463,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24410":{"gid":24410,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1529632960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1529632960},{"type":"v1","addr":"172.21.15.149:6839","nonce":1529632960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1429818440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1429818440},{"type":"v1","addr":"172.21.15.192:6839","nonce":1429818440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24457,24463]},"id":1}]} 2024-05-03T17:55:36.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 -- 172.21.15.122:0/2891843781 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbea00b2910 msgr2=0x7fbea00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:36.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbea00b2910 0x7fbea00b4dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8103040 tx=0x7fbeac009260 comp rx=0 tx=0).stop 2024-05-03T17:55:36.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 -- 172.21.15.122:0/2891843781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb8103750 msgr2=0x7fbeb819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:36.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb8103750 0x7fbeb819f6d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbea800d980 tx=0x7fbea800dc90 comp rx=0 tx=0).stop 2024-05-03T17:55:36.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 -- 172.21.15.122:0/2891843781 shutdown_connections 2024-05-03T17:55:36.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbea00b2910 0x7fbea00b4dd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:36.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8108850 0x7fbeb81998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:36.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb8103750 0x7fbeb819f6d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:36.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 --2- 172.21.15.122:0/2891843781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb8104100 0x7fbeb819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:36.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 -- 172.21.15.122:0/2891843781 >> 172.21.15.122:0/2891843781 conn(0x7fbeb807a600 msgr2=0x7fbeb8101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:36.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 -- 172.21.15.122:0/2891843781 shutdown_connections 2024-05-03T17:55:36.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:36.904+0000 7fbebd30b700 1 -- 172.21.15.122:0/2891843781 wait complete. 2024-05-03T17:55:36.900 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 24 2024-05-03T17:55:37.608 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T17:55:37.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:37 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2891843781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:37.847 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:37.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:37 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2891843781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:37.956 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:37 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2891843781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:38.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:38 smithi122 ceph-mon[29744]: pgmap v378: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:38.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:38 smithi149 ceph-mon[33409]: pgmap v378: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:38.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:38 smithi192 ceph-mon[33753]: pgmap v378: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:39.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.507+0000 7f4f0dadb700 1 -- 172.21.15.122:0/3667115751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f0806ba10 msgr2=0x7f4f0806be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:39.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.507+0000 7f4f0dadb700 1 --2- 172.21.15.122:0/3667115751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f0806ba10 0x7f4f0806be90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009990 tx=0x7f4efc009ca0 comp rx=0 tx=0).stop 2024-05-03T17:55:39.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.507+0000 7f4f0dadb700 1 -- 172.21.15.122:0/3667115751 shutdown_connections 2024-05-03T17:55:39.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.507+0000 7f4f0dadb700 1 --2- 172.21.15.122:0/3667115751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f0806c3d0 0x7f4f08110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:39.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.507+0000 7f4f0dadb700 1 --2- 172.21.15.122:0/3667115751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f08109500 0x7f4f081098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:39.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.507+0000 7f4f0dadb700 1 --2- 172.21.15.122:0/3667115751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f0806ba10 0x7f4f0806be90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:39.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.507+0000 7f4f0dadb700 1 -- 172.21.15.122:0/3667115751 >> 172.21.15.122:0/3667115751 conn(0x7f4f08079310 msgr2=0x7f4f08079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:39.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.507+0000 7f4f0dadb700 1 -- 172.21.15.122:0/3667115751 shutdown_connections 2024-05-03T17:55:39.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.507+0000 7f4f0dadb700 1 -- 172.21.15.122:0/3667115751 wait complete. 2024-05-03T17:55:39.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.508+0000 7f4f0dadb700 1 Processor -- start 2024-05-03T17:55:39.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.508+0000 7f4f0dadb700 1 -- start start 2024-05-03T17:55:39.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.508+0000 7f4f0dadb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f0806ba10 0x7f4f08102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:39.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.508+0000 7f4f0dadb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f0806c3d0 0x7f4f081029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.508+0000 7f4f0dadb700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f08109500 0x7f4f08102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.508+0000 7f4f0dadb700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08113690 con 0x7f4f08109500 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.508+0000 7f4f0dadb700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08113510 con 0x7f4f0806c3d0 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.508+0000 7f4f0dadb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08113810 con 0x7f4f0806ba10 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.508+0000 7f4f07fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f08109500 0x7f4f08102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.508+0000 7f4f07fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f08109500 0x7f4f08102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60198/0 (socket says 172.21.15.122:60198) 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.508+0000 7f4f07fff700 1 -- 172.21.15.122:0/873266560 learned_addr learned my addr 172.21.15.122:0/873266560 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f06ffd700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f0806c3d0 0x7f4f081029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f077fe700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f0806ba10 0x7f4f08102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f07fff700 1 -- 172.21.15.122:0/873266560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f0806ba10 msgr2=0x7f4f08102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f07fff700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f0806ba10 0x7f4f08102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f07fff700 1 -- 172.21.15.122:0/873266560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f0806c3d0 msgr2=0x7f4f081029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:39.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f07fff700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f0806c3d0 0x7f4f081029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:39.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f07fff700 1 -- 172.21.15.122:0/873266560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f081a80b0 con 0x7f4f08109500 2024-05-03T17:55:39.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f06ffd700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f0806c3d0 0x7f4f081029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:39.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f07fff700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f08109500 0x7f4f08102ee0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000b970 tx=0x7f4f0000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:39.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f04ff9700 1 -- 172.21.15.122:0/873266560 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00019070 con 0x7f4f08109500 2024-05-03T17:55:39.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f04ff9700 1 -- 172.21.15.122:0/873266560 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f00014440 con 0x7f4f08109500 2024-05-03T17:55:39.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f0dadb700 1 -- 172.21.15.122:0/873266560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f081a83a0 con 0x7f4f08109500 2024-05-03T17:55:39.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f04ff9700 1 -- 172.21.15.122:0/873266560 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000ce60 con 0x7f4f08109500 2024-05-03T17:55:39.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.509+0000 7f4f0dadb700 1 -- 172.21.15.122:0/873266560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f081a8890 con 0x7f4f08109500 2024-05-03T17:55:39.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.510+0000 7f4f0dadb700 1 -- 172.21.15.122:0/873266560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0806a190 con 0x7f4f08109500 2024-05-03T17:55:39.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.511+0000 7f4f04ff9700 1 -- 172.21.15.122:0/873266560 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f4f0001c070 con 0x7f4f08109500 2024-05-03T17:55:39.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.512+0000 7f4f04ff9700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4ef00b2910 0x7f4ef00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:39.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.512+0000 7f4f077fe700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4ef00b2910 0x7f4ef00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:39.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.512+0000 7f4f04ff9700 1 -- 172.21.15.122:0/873266560 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f4f000e1ab0 con 0x7f4f08109500 2024-05-03T17:55:39.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.512+0000 7f4f077fe700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4ef00b2910 0x7f4ef00b4dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8005ea0 tx=0x7f4ef8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:39.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.516+0000 7f4f04ff9700 1 -- 172.21.15.122:0/873266560 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4f000a9880 con 0x7f4f08109500 2024-05-03T17:55:39.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.766+0000 7f4f0dadb700 1 -- 172.21.15.122:0/873266560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f08103990 con 0x7f4f08109500 2024-05-03T17:55:39.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.767+0000 7f4f04ff9700 1 -- 172.21.15.122:0/873266560 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11892 (secure 0 0 0) 0x7f4f000a9180 con 0x7f4f08109500 2024-05-03T17:55:39.762 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:55:39.762 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/555236113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":555236113},{"type":"v1","addr":"172.21.15.122:6839","nonce":555236113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1783292878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1783292878},{"type":"v1","addr":"172.21.15.122:6841","nonce":1783292878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T17:52:37.616728+0000","modified":"2024-05-03T17:52:56.867078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24463,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24410":{"gid":24410,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1529632960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1529632960},{"type":"v1","addr":"172.21.15.149:6839","nonce":1529632960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1429818440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1429818440},{"type":"v1","addr":"172.21.15.192:6839","nonce":1429818440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24457,24463]},"id":1}]} 2024-05-03T17:55:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.769+0000 7f4f0dadb700 1 -- 172.21.15.122:0/873266560 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4ef00b2910 msgr2=0x7f4ef00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.769+0000 7f4f0dadb700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4ef00b2910 0x7f4ef00b4dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8005ea0 tx=0x7f4ef8005e30 comp rx=0 tx=0).stop 2024-05-03T17:55:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.769+0000 7f4f0dadb700 1 -- 172.21.15.122:0/873266560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f08109500 msgr2=0x7f4f08102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.769+0000 7f4f0dadb700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f08109500 0x7f4f08102ee0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000b970 tx=0x7f4f0000bd30 comp rx=0 tx=0).stop 2024-05-03T17:55:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.769+0000 7f4f0dadb700 1 -- 172.21.15.122:0/873266560 shutdown_connections 2024-05-03T17:55:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.769+0000 7f4f0dadb700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f4ef00b2910 0x7f4ef00b4dd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.769+0000 7f4f0dadb700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f08109500 0x7f4f08102ee0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.769+0000 7f4f0dadb700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f0806ba10 0x7f4f08102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.769+0000 7f4f0dadb700 1 --2- 172.21.15.122:0/873266560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f0806c3d0 0x7f4f081029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.769+0000 7f4f0dadb700 1 -- 172.21.15.122:0/873266560 >> 172.21.15.122:0/873266560 conn(0x7f4f08079310 msgr2=0x7f4f08101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.770+0000 7f4f0dadb700 1 -- 172.21.15.122:0/873266560 shutdown_connections 2024-05-03T17:55:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:39.770+0000 7f4f0dadb700 1 -- 172.21.15.122:0/873266560 wait complete. 2024-05-03T17:55:39.766 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 24 2024-05-03T17:55:40.419 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T17:55:40.660 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:40.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:40 smithi122 ceph-mon[29744]: pgmap v379: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:40.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:40 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/873266560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:40.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:40 smithi149 ceph-mon[33409]: pgmap v379: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:40.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:40 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/873266560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:40.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:40 smithi192 ceph-mon[33753]: pgmap v379: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:40.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:40 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/873266560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T17:55:42.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.373+0000 7f254e8ac700 1 -- 172.21.15.122:0/1432085409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2548109520 msgr2=0x7f2548109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:42.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.373+0000 7f254e8ac700 1 --2- 172.21.15.122:0/1432085409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2548109520 0x7f2548109900 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2530009990 tx=0x7f2530009ca0 comp rx=0 tx=0).stop 2024-05-03T17:55:42.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.373+0000 7f254e8ac700 1 -- 172.21.15.122:0/1432085409 shutdown_connections 2024-05-03T17:55:42.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.373+0000 7f254e8ac700 1 --2- 172.21.15.122:0/1432085409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2548102450 0x7f2548110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:42.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.373+0000 7f254e8ac700 1 --2- 172.21.15.122:0/1432085409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2548109520 0x7f2548109900 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:42.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.373+0000 7f254e8ac700 1 --2- 172.21.15.122:0/1432085409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2548101a90 0x7f2548101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:42.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.373+0000 7f254e8ac700 1 -- 172.21.15.122:0/1432085409 >> 172.21.15.122:0/1432085409 conn(0x7f2548078a10 msgr2=0x7f25480ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:42.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.373+0000 7f254e8ac700 1 -- 172.21.15.122:0/1432085409 shutdown_connections 2024-05-03T17:55:42.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.373+0000 7f254e8ac700 1 -- 172.21.15.122:0/1432085409 wait complete. 2024-05-03T17:55:42.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f254e8ac700 1 Processor -- start 2024-05-03T17:55:42.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f254e8ac700 1 -- start start 2024-05-03T17:55:42.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f254e8ac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2548101a90 0x7f2548102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:42.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f254e8ac700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2548102450 0x7f25481031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:42.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f254e8ac700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2548109520 0x7f25481036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:42.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f254e8ac700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2548113730 con 0x7f2548102450 2024-05-03T17:55:42.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f254e8ac700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25481135b0 con 0x7f2548109520 2024-05-03T17:55:42.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f254e8ac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f25481138b0 con 0x7f2548101a90 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f25477fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2548102450 0x7f25481031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f25477fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2548102450 0x7f25481031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60226/0 (socket says 172.21.15.122:60226) 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f25477fe700 1 -- 172.21.15.122:0/263379466 learned_addr learned my addr 172.21.15.122:0/263379466 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f254ce49700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2548109520 0x7f25481036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.374+0000 7f2547fff700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2548101a90 0x7f2548102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.375+0000 7f25477fe700 1 -- 172.21.15.122:0/263379466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2548101a90 msgr2=0x7f2548102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.375+0000 7f25477fe700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2548101a90 0x7f2548102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.375+0000 7f25477fe700 1 -- 172.21.15.122:0/263379466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2548109520 msgr2=0x7f25481036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.375+0000 7f25477fe700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2548109520 0x7f25481036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.375+0000 7f25477fe700 1 -- 172.21.15.122:0/263379466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25481a81f0 con 0x7f2548102450 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.375+0000 7f25477fe700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2548102450 0x7f25481031b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f253800b530 tx=0x7f253800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.375+0000 7f25457fa700 1 -- 172.21.15.122:0/263379466 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538015020 con 0x7f2548102450 2024-05-03T17:55:42.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.375+0000 7f25457fa700 1 -- 172.21.15.122:0/263379466 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2538011ce0 con 0x7f2548102450 2024-05-03T17:55:42.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.375+0000 7f254e8ac700 1 -- 172.21.15.122:0/263379466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25481a84e0 con 0x7f2548102450 2024-05-03T17:55:42.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.375+0000 7f25457fa700 1 -- 172.21.15.122:0/263379466 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538010e40 con 0x7f2548102450 2024-05-03T17:55:42.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.375+0000 7f254e8ac700 1 -- 172.21.15.122:0/263379466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25481a8a20 con 0x7f2548102450 2024-05-03T17:55:42.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.376+0000 7f254e8ac700 1 -- 172.21.15.122:0/263379466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254810ba00 con 0x7f2548102450 2024-05-03T17:55:42.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.377+0000 7f25457fa700 1 -- 172.21.15.122:0/263379466 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2538018020 con 0x7f2548102450 2024-05-03T17:55:42.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.378+0000 7f25457fa700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25340b2910 0x7f25340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:42.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.378+0000 7f2547fff700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25340b2910 0x7f25340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:42.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.378+0000 7f25457fa700 1 -- 172.21.15.122:0/263379466 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f25380e1f80 con 0x7f2548102450 2024-05-03T17:55:42.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.378+0000 7f2547fff700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25340b2910 0x7f25340b4dd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2530006010 tx=0x7f253001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:42.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.381+0000 7f25457fa700 1 -- 172.21.15.122:0/263379466 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f253801f020 con 0x7f2548102450 2024-05-03T17:55:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.588+0000 7f254e8ac700 1 -- 172.21.15.122:0/263379466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f254810b3e0 con 0x7f2548102450 2024-05-03T17:55:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.589+0000 7f25457fa700 1 -- 172.21.15.122:0/263379466 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22622 (secure 0 0 0) 0x7f25380a9d50 con 0x7f2548102450 2024-05-03T17:55:42.586 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T17:55:42.586 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":79,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","created":"2024-05-03T17:44:17.415571+0000","modified":"2024-05-03T17:53:19.863832+0000","last_up_change":"2024-05-03T17:51:28.839117+0000","last_in_change":"2024-05-03T17:51:07.954070+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T17:47:40.539967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T17:52:31.646919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T17:52:34.503067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4b08628-0a29-44aa-b8b9-ed9657f12be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6803","nonce":1329364610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6805","nonce":1329364610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6809","nonce":1329364610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6807","nonce":1329364610}]},"public_addr":"172.21.15.122:6803/1329364610","cluster_addr":"172.21.15.122:6805/1329364610","heartbeat_back_addr":"172.21.15.122:6809/1329364610","heartbeat_front_addr":"172.21.15.122:6807/1329364610","state":["exists","up"]},{"osd":1,"uuid":"ba550787-ce76-4944-8ebd-97e19590bb3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6801","nonce":1402463302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6803","nonce":1402463302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6807","nonce":1402463302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6805","nonce":1402463302}]},"public_addr":"172.21.15.149:6801/1402463302","cluster_addr":"172.21.15.149:6803/1402463302","heartbeat_back_addr":"172.21.15.149:6807/1402463302","heartbeat_front_addr":"172.21.15.149:6805/1402463302","state":["exists","up"]},{"osd":2,"uuid":"35ae7c47-d710-47c5-b0ec-91fa12b5be07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6801","nonce":3757191110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6803","nonce":3757191110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6807","nonce":3757191110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6805","nonce":3757191110}]},"public_addr":"172.21.15.192:6801/3757191110","cluster_addr":"172.21.15.192:6803/3757191110","heartbeat_back_addr":"172.21.15.192:6807/3757191110","heartbeat_front_addr":"172.21.15.192:6805/3757191110","state":["exists","up"]},{"osd":3,"uuid":"9973c0c1-ae73-497a-845e-664a20666df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6811","nonce":2359143764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6813","nonce":2359143764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6817","nonce":2359143764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6815","nonce":2359143764}]},"public_addr":"172.21.15.122:6811/2359143764","cluster_addr":"172.21.15.122:6813/2359143764","heartbeat_back_addr":"172.21.15.122:6817/2359143764","heartbeat_front_addr":"172.21.15.122:6815/2359143764","state":["exists","up"]},{"osd":4,"uuid":"da71dc18-94b9-4f7d-a061-7cc7b77db282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3627037532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3627037532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3627037532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3627037532}]},"public_addr":"172.21.15.149:6809/3627037532","cluster_addr":"172.21.15.149:6811/3627037532","heartbeat_back_addr":"172.21.15.149:6815/3627037532","heartbeat_front_addr":"172.21.15.149:6813/3627037532","state":["exists","up"]},{"osd":5,"uuid":"bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6809","nonce":455779485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6811","nonce":455779485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6815","nonce":455779485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6813","nonce":455779485}]},"public_addr":"172.21.15.192:6809/455779485","cluster_addr":"172.21.15.192:6811/455779485","heartbeat_back_addr":"172.21.15.192:6815/455779485","heartbeat_front_addr":"172.21.15.192:6813/455779485","state":["exists","up"]},{"osd":6,"uuid":"4150bfe5-a0e4-4bae-914a-75598f017519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6819","nonce":3419421998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6821","nonce":3419421998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6825","nonce":3419421998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6823","nonce":3419421998}]},"public_addr":"172.21.15.122:6819/3419421998","cluster_addr":"172.21.15.122:6821/3419421998","heartbeat_back_addr":"172.21.15.122:6825/3419421998","heartbeat_front_addr":"172.21.15.122:6823/3419421998","state":["exists","up"]},{"osd":7,"uuid":"71e01812-e511-44a0-a143-2afca510ae8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6817","nonce":139747549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6819","nonce":139747549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6823","nonce":139747549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6821","nonce":139747549}]},"public_addr":"172.21.15.149:6817/139747549","cluster_addr":"172.21.15.149:6819/139747549","heartbeat_back_addr":"172.21.15.149:6823/139747549","heartbeat_front_addr":"172.21.15.149:6821/139747549","state":["exists","up"]},{"osd":8,"uuid":"5294516d-fac6-41d0-a635-6bc1d7eca585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6817","nonce":2080816744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6819","nonce":2080816744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6823","nonce":2080816744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6821","nonce":2080816744}]},"public_addr":"172.21.15.192:6817/2080816744","cluster_addr":"172.21.15.192:6819/2080816744","heartbeat_back_addr":"172.21.15.192:6823/2080816744","heartbeat_front_addr":"172.21.15.192:6821/2080816744","state":["exists","up"]},{"osd":9,"uuid":"f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6827","nonce":1773691421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6829","nonce":1773691421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6833","nonce":1773691421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6831","nonce":1773691421}]},"public_addr":"172.21.15.122:6827/1773691421","cluster_addr":"172.21.15.122:6829/1773691421","heartbeat_back_addr":"172.21.15.122:6833/1773691421","heartbeat_front_addr":"172.21.15.122:6831/1773691421","state":["exists","up"]},{"osd":10,"uuid":"af88e50f-98f1-460d-856b-a920dd2ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6825","nonce":1954186137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6827","nonce":1954186137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6831","nonce":1954186137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6829","nonce":1954186137}]},"public_addr":"172.21.15.149:6825/1954186137","cluster_addr":"172.21.15.149:6827/1954186137","heartbeat_back_addr":"172.21.15.149:6831/1954186137","heartbeat_front_addr":"172.21.15.149:6829/1954186137","state":["exists","up"]},{"osd":11,"uuid":"b54c1b4d-cdb5-42d9-979a-47f9a870b5cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6825","nonce":2098474209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6827","nonce":2098474209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6831","nonce":2098474209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6829","nonce":2098474209}]},"public_addr":"172.21.15.192:6825/2098474209","cluster_addr":"172.21.15.192:6827/2098474209","heartbeat_back_addr":"172.21.15.192:6831/2098474209","heartbeat_front_addr":"172.21.15.192:6829/2098474209","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:46:50.049769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:13.518485+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:37.473023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:01.809640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:26.648514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:51.268056+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:16.294834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:42.079065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:07.609271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:33.885127+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:59.982708+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:51:26.461369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/2089907395":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6801/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:0/2001860625":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/1888284988":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/256574472":"2024-05-04T17:44:52.244049+0000","172.21.15.122:6801/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/64546794":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/584846864":"2024-05-04T17:45:18.454406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T17:55:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 -- 172.21.15.122:0/263379466 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25340b2910 msgr2=0x7f25340b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25340b2910 0x7f25340b4dd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2530006010 tx=0x7f253001a040 comp rx=0 tx=0).stop 2024-05-03T17:55:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 -- 172.21.15.122:0/263379466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2548102450 msgr2=0x7f25481031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2548102450 0x7f25481031b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f253800b530 tx=0x7f253800b840 comp rx=0 tx=0).stop 2024-05-03T17:55:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 -- 172.21.15.122:0/263379466 shutdown_connections 2024-05-03T17:55:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f25340b2910 0x7f25340b4dd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2548109520 0x7f25481036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2548101a90 0x7f2548102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 --2- 172.21.15.122:0/263379466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2548102450 0x7f25481031b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 -- 172.21.15.122:0/263379466 >> 172.21.15.122:0/263379466 conn(0x7f2548078a10 msgr2=0x7f2548100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 -- 172.21.15.122:0/263379466 shutdown_connections 2024-05-03T17:55:42.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:42.592+0000 7f254e8ac700 1 -- 172.21.15.122:0/263379466 wait complete. 2024-05-03T17:55:42.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:42 smithi149 ceph-mon[33409]: pgmap v380: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:42.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:42 smithi192 ceph-mon[33753]: pgmap v380: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:42.906 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:42 smithi122 ceph-mon[29744]: pgmap v380: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:43.229 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-03T17:52:37.616728+0000', 'modified': '2024-05-03T17:52:56.867078+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24481, 'mds_1': 24463, 'mds_2': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24410': {'gid': 24410, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/2599524712', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 2599524712}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 2599524712}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6833/899389349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6832', 'nonce': 899389349}, {'type': 'v1', 'addr': '172.21.15.192:6833', 'nonce': 899389349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6839/1529632960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 1529632960}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 1529632960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.192:6835/2756708496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 2756708496}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 2756708496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.192:6837/3476407415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 3476407415}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 3476407415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.192:6839/1429818440', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 1429818440}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 1429818440}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24481, 'qdb_cluster': [24481, 24457, 24463]} 2024-05-03T17:55:43.229 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-03T17:55:43.229 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-03T17:55:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:43.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:43 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/263379466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:55:43.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:43 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/263379466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:55:43.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:43 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/263379466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T17:55:44.591 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:44 smithi122 ceph-mon[29744]: pgmap v381: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:44.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:44 smithi149 ceph-mon[33409]: pgmap v381: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:44.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:44 smithi192 ceph-mon[33753]: pgmap v381: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:45.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.117+0000 7f899e88e700 1 -- 172.21.15.122:0/2267096705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89981023a0 msgr2=0x7f8998102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:45.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.117+0000 7f899e88e700 1 --2- 172.21.15.122:0/2267096705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89981023a0 0x7f8998102820 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8988009990 tx=0x7f8988009ca0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.117+0000 7f899e88e700 1 -- 172.21.15.122:0/2267096705 shutdown_connections 2024-05-03T17:55:45.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.117+0000 7f899e88e700 1 --2- 172.21.15.122:0/2267096705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8998102d60 0x7f899810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.117+0000 7f899e88e700 1 --2- 172.21.15.122:0/2267096705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8998108410 0x7f89981087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.117+0000 7f899e88e700 1 --2- 172.21.15.122:0/2267096705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89981023a0 0x7f8998102820 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.117+0000 7f899e88e700 1 -- 172.21.15.122:0/2267096705 >> 172.21.15.122:0/2267096705 conn(0x7f8998079350 msgr2=0x7f8998079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:45.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.117+0000 7f899e88e700 1 -- 172.21.15.122:0/2267096705 shutdown_connections 2024-05-03T17:55:45.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.117+0000 7f899e88e700 1 -- 172.21.15.122:0/2267096705 wait complete. 2024-05-03T17:55:45.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f899e88e700 1 Processor -- start 2024-05-03T17:55:45.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f899e88e700 1 -- start start 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f899e88e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89981023a0 0x7f899819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f899e88e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8998102d60 0x7f899819ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f899e88e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8998108410 0x7f8998199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f899e88e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8998112680 con 0x7f89981023a0 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f899e88e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8998112500 con 0x7f8998108410 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f899e88e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8998112800 con 0x7f8998102d60 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f8997fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89981023a0 0x7f899819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f8997fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89981023a0 0x7f899819fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60250/0 (socket says 172.21.15.122:60250) 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f8997fff700 1 -- 172.21.15.122:0/4017469161 learned_addr learned my addr 172.21.15.122:0/4017469161 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f89977fe700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8998102d60 0x7f899819ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.118+0000 7f899ce2b700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8998108410 0x7f8998199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.119+0000 7f8997fff700 1 -- 172.21.15.122:0/4017469161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8998102d60 msgr2=0x7f899819ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.119+0000 7f8997fff700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8998102d60 0x7f899819ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.119+0000 7f8997fff700 1 -- 172.21.15.122:0/4017469161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8998108410 msgr2=0x7f8998199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:45.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.119+0000 7f8997fff700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8998108410 0x7f8998199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.119+0000 7f8997fff700 1 -- 172.21.15.122:0/4017469161 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899819a410 con 0x7f89981023a0 2024-05-03T17:55:45.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.119+0000 7f8997fff700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89981023a0 0x7f899819fa60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b530 tx=0x7f898c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:45.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.119+0000 7f89957fa700 1 -- 172.21.15.122:0/4017469161 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c015020 con 0x7f89981023a0 2024-05-03T17:55:45.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.119+0000 7f89957fa700 1 -- 172.21.15.122:0/4017469161 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898c011ce0 con 0x7f89981023a0 2024-05-03T17:55:45.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.119+0000 7f899e88e700 1 -- 172.21.15.122:0/4017469161 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899819a700 con 0x7f89981023a0 2024-05-03T17:55:45.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.119+0000 7f89957fa700 1 -- 172.21.15.122:0/4017469161 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c010dd0 con 0x7f89981023a0 2024-05-03T17:55:45.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.119+0000 7f899e88e700 1 -- 172.21.15.122:0/4017469161 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89981a69e0 con 0x7f89981023a0 2024-05-03T17:55:45.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.121+0000 7f89957fa700 1 -- 172.21.15.122:0/4017469161 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f898c018020 con 0x7f89981023a0 2024-05-03T17:55:45.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.122+0000 7f89957fa700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f89800b2910 0x7f89800b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.122+0000 7f89977fe700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f89800b2910 0x7f89800b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.122+0000 7f89957fa700 1 -- 172.21.15.122:0/4017469161 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f898c0e1850 con 0x7f89981023a0 2024-05-03T17:55:45.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.122+0000 7f89977fe700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f89800b2910 0x7f89800b4dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f899819b280 tx=0x7f8988005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:45.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.122+0000 7f89a9121b80 1 Processor -- start 2024-05-03T17:55:45.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.122+0000 7f89a9121b80 1 -- start start 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e9740 0x5613bd7e9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7ea060 0x5613bd7e4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e5480 0x5613bd7e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f89a9121b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613bd7378f0 con 0x5613bd7e9740 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f89a9121b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5613bd69cd50 con 0x5613bd7ea060 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f89a9121b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5613bd7338b0 con 0x5613bd7e5480 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f8997fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e9740 0x5613bd7e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f8997fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e9740 0x5613bd7e9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60264/0 (socket says 172.21.15.122:60264) 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f8997fff700 1 -- 172.21.15.122:0/79521785 learned_addr learned my addr 172.21.15.122:0/79521785 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f899ce2b700 1 --2- 172.21.15.122:0/79521785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e5480 0x5613bd7e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f8997fff700 1 -- 172.21.15.122:0/79521785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e5480 msgr2=0x5613bd7e58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f8997fff700 1 --2- 172.21.15.122:0/79521785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e5480 0x5613bd7e58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f8997fff700 1 -- 172.21.15.122:0/79521785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7ea060 msgr2=0x5613bd7e4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f8997fff700 1 --2- 172.21.15.122:0/79521785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7ea060 0x5613bd7e4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f8997fff700 1 -- 172.21.15.122:0/79521785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613bd684880 con 0x5613bd7e9740 2024-05-03T17:55:45.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f8997fff700 1 --2- 172.21.15.122:0/79521785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e9740 0x5613bd7e9b20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f898c0a4fa0 tx=0x7f898c0a8690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5959e0f2d218c17e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:45.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f897e7fc700 1 -- 172.21.15.122:0/79521785 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c0a8f20 con 0x5613bd7e9740 2024-05-03T17:55:45.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f897e7fc700 1 -- 172.21.15.122:0/79521785 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898c0ad2d0 con 0x5613bd7e9740 2024-05-03T17:55:45.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.123+0000 7f897e7fc700 1 -- 172.21.15.122:0/79521785 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c0a83c0 con 0x5613bd7e9740 2024-05-03T17:55:45.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.124+0000 7f89a9121b80 1 -- 172.21.15.122:0/79521785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e9740 msgr2=0x5613bd7e9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.124+0000 7f89a9121b80 1 --2- 172.21.15.122:0/79521785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e9740 0x5613bd7e9b20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f898c0a4fa0 tx=0x7f898c0a8690 comp rx=0 tx=0).stop 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.124+0000 7f89a9121b80 1 -- 172.21.15.122:0/79521785 shutdown_connections 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.124+0000 7f89a9121b80 1 --2- 172.21.15.122:0/79521785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e5480 0x5613bd7e58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.124+0000 7f89a9121b80 1 --2- 172.21.15.122:0/79521785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e9740 0x5613bd7e9b20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.124+0000 7f89a9121b80 1 --2- 172.21.15.122:0/79521785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7ea060 0x5613bd7e4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.124+0000 7f89a9121b80 1 -- 172.21.15.122:0/79521785 >> 172.21.15.122:0/79521785 conn(0x5613bd711b20 msgr2=0x5613bd7df9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.124+0000 7f89a9121b80 1 -- 172.21.15.122:0/79521785 shutdown_connections 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.124+0000 7f89a9121b80 1 -- 172.21.15.122:0/79521785 wait complete. 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.124+0000 7f89a9121b80 1 -- start start 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f89a9121b80 1 Processor -- start 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e5480 0x5613bd7f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e9740 0x5613bd7ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7ea060 0x5613bd7ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f89a9121b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613bd7f1ba0 con 0x5613bd7e5480 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f89a9121b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5613bd6f0e80 con 0x5613bd7e9740 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f89a9121b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5613bd7f1d20 con 0x5613bd7ea060 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f89a9121b80 10 client.0 _ll_register_callbacks cb 0x5613bd6cb560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f8997fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e5480 0x5613bd7f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f8997fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e5480 0x5613bd7f0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60274/0 (socket says 172.21.15.122:60274) 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f8997fff700 1 -- 172.21.15.122:0/3002127703 learned_addr learned my addr 172.21.15.122:0/3002127703 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f89977fe700 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e9740 0x5613bd7ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f8997fff700 1 -- 172.21.15.122:0/3002127703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7ea060 msgr2=0x5613bd7ee770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f8997fff700 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7ea060 0x5613bd7ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f8997fff700 1 -- 172.21.15.122:0/3002127703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e9740 msgr2=0x5613bd7ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f8997fff700 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e9740 0x5613bd7ee230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f8997fff700 1 -- 172.21.15.122:0/3002127703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613bd6ee710 con 0x5613bd7e5480 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f8997fff700 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e5480 0x5613bd7f0e90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f898c0a8c20 tx=0x7f898c0b1bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd16d38556da4ad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.125+0000 7f89777fe700 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89777fe700 1 -- 172.21.15.122:0/3002127703 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c06d070 con 0x5613bd7e5480 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89777fe700 1 -- 172.21.15.122:0/3002127703 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f898c0a5510 con 0x5613bd7e5480 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89a9121b80 1 -- 172.21.15.122:0/3002127703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f898c0ad2d0 con 0x5613bd7e5480 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89777fe700 1 -- 172.21.15.122:0/3002127703 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f898c0b1d80 con 0x5613bd7e5480 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89a9121b80 10 client.15126 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89a9121b80 1 -- 172.21.15.122:0/3002127703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5613bd7f1ba0 con 0x5613bd7e5480 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89777fe700 1 -- 172.21.15.122:0/3002127703 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898c0ad2d0 con 0x5613bd7e5480 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89777fe700 1 -- 172.21.15.122:0/3002127703 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f898c0ac030 con 0x5613bd7e5480 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89a9121b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89a9121b80 10 client.15126 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89a9121b80 20 client.15126 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89a9121b80 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x5613bd7fe4b0 0x5613bd7ff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89a9121b80 4 client.15126 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.126+0000 7f89a9121b80 1 -- 172.21.15.122:0/3002127703 --> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613bd7452d0 con 0x5613bd7fe4b0 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.127+0000 7f89977fe700 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x5613bd7fe4b0 0x5613bd7ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.127+0000 7f89977fe700 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x5613bd7fe4b0 0x5613bd7ff920 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1da120ff047666c1 server_cookie=e20950321abe7e72 in_seq=0 out_seq=0 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.127+0000 7f89777fe700 10 client.15126 ms_handle_connect on v2:172.21.15.192:6834/2756708496 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89777fe700 1 -- 172.21.15.122:0/3002127703 <== mds.2 v2:172.21.15.192:6834/2756708496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5613bd7452d0 con 0x5613bd7fe4b0 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89777fe700 10 client.15126 handle_command_reply: tid=0 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 1 client.15126 shutdown 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 1 -- 172.21.15.122:0/3002127703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e5480 msgr2=0x5613bd7f0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e5480 0x5613bd7f0e90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f898c0a8c20 tx=0x7f898c0b1bd0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 1 -- 172.21.15.122:0/3002127703 shutdown_connections 2024-05-03T17:55:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x5613bd7fe4b0 0x5613bd7ff920 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7ea060 0x5613bd7ee770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7e5480 0x5613bd7f0e90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89777fe700 0 client.15126 ms_handle_reset on v2:172.21.15.192:6834/2756708496 2024-05-03T17:55:45.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 1 --2- 172.21.15.122:0/3002127703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e9740 0x5613bd7ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 1 -- 172.21.15.122:0/3002127703 >> 172.21.15.122:0/3002127703 conn(0x5613bd711b20 msgr2=0x5613bd7de800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:45.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 1 -- 172.21.15.122:0/3002127703 shutdown_connections 2024-05-03T17:55:45.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 1 -- 172.21.15.122:0/3002127703 wait complete. 2024-05-03T17:55:45.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.128+0000 7f89a9121b80 20 client.15126 trim_cache size 0 max 16384 2024-05-03T17:55:45.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f89a9121b80 1 Processor -- start 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f89a9121b80 1 -- start start 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e5480 0x5613bd7df810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e9740 0x5613bd7e8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd7eb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f89a9121b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613bd7fdc60 con 0x5613bd7ea060 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f89a9121b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5613bd7fe330 con 0x5613bd7e9740 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f89a9121b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5613bd7fea00 con 0x5613bd7e5480 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f899ce2b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd7eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f899ce2b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd7eb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60280/0 (socket says 172.21.15.122:60280) 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f899ce2b700 1 -- 172.21.15.122:0/587398487 learned_addr learned my addr 172.21.15.122:0/587398487 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f89977fe700 1 --2- 172.21.15.122:0/587398487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e9740 0x5613bd7e8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f8997fff700 1 --2- 172.21.15.122:0/587398487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e5480 0x5613bd7df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f899ce2b700 1 -- 172.21.15.122:0/587398487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e5480 msgr2=0x5613bd7df810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f899ce2b700 1 --2- 172.21.15.122:0/587398487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e5480 0x5613bd7df810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.149+0000 7f899ce2b700 1 -- 172.21.15.122:0/587398487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e9740 msgr2=0x5613bd7e8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.150+0000 7f899ce2b700 1 --2- 172.21.15.122:0/587398487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e9740 0x5613bd7e8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.150+0000 7f899ce2b700 1 -- 172.21.15.122:0/587398487 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613bd7f1470 con 0x5613bd7ea060 2024-05-03T17:55:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.150+0000 7f8997fff700 1 --2- 172.21.15.122:0/587398487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e5480 0x5613bd7df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:55:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.150+0000 7f89977fe700 1 --2- 172.21.15.122:0/587398487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e9740 0x5613bd7e8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.150+0000 7f899ce2b700 1 --2- 172.21.15.122:0/587398487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd7eb450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8990009f50 tx=0x7f899000a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9c261a58a6665c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.150+0000 7f897dffb700 1 -- 172.21.15.122:0/587398487 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89900155f0 con 0x5613bd7ea060 2024-05-03T17:55:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.150+0000 7f897dffb700 1 -- 172.21.15.122:0/587398487 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f899001a070 con 0x5613bd7ea060 2024-05-03T17:55:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.150+0000 7f897dffb700 1 -- 172.21.15.122:0/587398487 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89900149c0 con 0x5613bd7ea060 2024-05-03T17:55:45.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 -- 172.21.15.122:0/587398487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 msgr2=0x5613bd7eb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:45.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 --2- 172.21.15.122:0/587398487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd7eb450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8990009f50 tx=0x7f899000a6e0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 -- 172.21.15.122:0/587398487 shutdown_connections 2024-05-03T17:55:45.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 --2- 172.21.15.122:0/587398487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd7eb450 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 --2- 172.21.15.122:0/587398487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e5480 0x5613bd7df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 --2- 172.21.15.122:0/587398487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e9740 0x5613bd7e8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 -- 172.21.15.122:0/587398487 >> 172.21.15.122:0/587398487 conn(0x5613bd711b20 msgr2=0x5613bd7e3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:45.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 -- 172.21.15.122:0/587398487 shutdown_connections 2024-05-03T17:55:45.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 -- 172.21.15.122:0/587398487 wait complete. 2024-05-03T17:55:45.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 -- start start 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 Processor -- start 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e5480 0x5613bd802cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e9740 0x5613bd803230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd8074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613bd7ff7a0 con 0x5613bd7ea060 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5613bd7ff0d0 con 0x5613bd7e5480 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5613bd7ffe70 con 0x5613bd7e9740 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.151+0000 7f89a9121b80 10 client.0 _ll_register_callbacks cb 0x5613bd6cb560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f899ce2b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd8074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f899ce2b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd8074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60292/0 (socket says 172.21.15.122:60292) 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f899ce2b700 1 -- 172.21.15.122:0/2632948393 learned_addr learned my addr 172.21.15.122:0/2632948393 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f89977fe700 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e9740 0x5613bd803230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f8997fff700 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e5480 0x5613bd802cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f899ce2b700 1 -- 172.21.15.122:0/2632948393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e9740 msgr2=0x5613bd803230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f899ce2b700 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e9740 0x5613bd803230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f899ce2b700 1 -- 172.21.15.122:0/2632948393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e5480 msgr2=0x5613bd802cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f899ce2b700 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e5480 0x5613bd802cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f899ce2b700 1 -- 172.21.15.122:0/2632948393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613bd8079f0 con 0x5613bd7ea060 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f899ce2b700 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd8074b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f89900092d0 tx=0x7f8990007070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9064c4614dc98cd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f89777fe700 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f89777fe700 1 -- 172.21.15.122:0/2632948393 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8990014350 con 0x5613bd7ea060 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f89777fe700 1 -- 172.21.15.122:0/2632948393 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f899002d480 con 0x5613bd7ea060 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f89a9121b80 1 -- 172.21.15.122:0/2632948393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f899001a070 con 0x5613bd7ea060 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f89777fe700 1 -- 172.21.15.122:0/2632948393 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f899002d680 con 0x5613bd7ea060 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f89a9121b80 10 client.15132 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.152+0000 7f89a9121b80 1 -- 172.21.15.122:0/2632948393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5613bd869920 con 0x5613bd7ea060 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.153+0000 7f89777fe700 1 -- 172.21.15.122:0/2632948393 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f899001a070 con 0x5613bd7ea060 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.153+0000 7f89777fe700 1 -- 172.21.15.122:0/2632948393 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8990038880 con 0x5613bd7ea060 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.153+0000 7f89a9121b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.153+0000 7f89a9121b80 10 client.15132 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.153+0000 7f89a9121b80 20 client.15132 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.153+0000 7f89a9121b80 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x5613bd86eb40 0x5613bd86d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.153+0000 7f89a9121b80 4 client.15132 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.153+0000 7f89a9121b80 1 -- 172.21.15.122:0/2632948393 --> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] -- command(tid 0: {"prefix": "status"}) -- 0x5613bd73f370 con 0x5613bd86eb40 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.153+0000 7f8997fff700 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x5613bd86eb40 0x5613bd86d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f8997fff700 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x5613bd86eb40 0x5613bd86d340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7613ffdf5db91678 server_cookie=2e421fc1b55de52e in_seq=0 out_seq=0 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89777fe700 10 client.15132 ms_handle_connect on v2:172.21.15.192:6834/2756708496 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89777fe700 1 -- 172.21.15.122:0/2632948393 <== mds.2 v2:172.21.15.192:6834/2756708496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5613bd73f370 con 0x5613bd86eb40 2024-05-03T17:55:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89777fe700 10 client.15132 handle_command_reply: tid=0 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 1 client.15132 shutdown 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 1 -- 172.21.15.122:0/2632948393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 msgr2=0x5613bd8074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd8074b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f89900092d0 tx=0x7f8990007070 comp rx=0 tx=0).stop 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 1 -- 172.21.15.122:0/2632948393 shutdown_connections 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.192:6834/2756708496,v1:172.21.15.192:6835/2756708496] conn(0x5613bd86eb40 0x5613bd86d340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613bd7ea060 0x5613bd8074b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613bd7e5480 0x5613bd802cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89777fe700 0 client.15132 ms_handle_reset on v2:172.21.15.192:6834/2756708496 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 1 --2- 172.21.15.122:0/2632948393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5613bd7e9740 0x5613bd803230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 1 -- 172.21.15.122:0/2632948393 >> 172.21.15.122:0/2632948393 conn(0x5613bd711b20 msgr2=0x5613bd7e8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 1 -- 172.21.15.122:0/2632948393 shutdown_connections 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 1 -- 172.21.15.122:0/2632948393 wait complete. 2024-05-03T17:55:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.154+0000 7f89a9121b80 20 client.15132 trim_cache size 0 max 16384 2024-05-03T17:55:45.156 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-05-03T17:55:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "a46fc846-0974-11ef-bc97-c7b262605968", 2024-05-03T17:55:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 2, 2024-05-03T17:55:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24457, 2024-05-03T17:55:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-05-03T17:55:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-05-03T17:55:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-05-03T17:55:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 174.376474078, 2024-05-03T17:55:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 24, 2024-05-03T17:55:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 79, 2024-05-03T17:55:45.156 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 77, 2024-05-03T17:55:45.157 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 222.97630672099999 2024-05-03T17:55:45.157 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-05-03T17:55:45.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 -- 172.21.15.122:0/4017469161 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f89800b2910 msgr2=0x7f89800b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:45.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f89800b2910 0x7f89800b4dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f899819b280 tx=0x7f8988005fd0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 -- 172.21.15.122:0/4017469161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89981023a0 msgr2=0x7f899819fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:45.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89981023a0 0x7f899819fa60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b530 tx=0x7f898c00b840 comp rx=0 tx=0).stop 2024-05-03T17:55:45.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 -- 172.21.15.122:0/4017469161 shutdown_connections 2024-05-03T17:55:45.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f89800b2910 0x7f89800b4dd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8998108410 0x7f8998199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89981023a0 0x7f899819fa60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 --2- 172.21.15.122:0/4017469161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8998102d60 0x7f899819ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:45.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 -- 172.21.15.122:0/4017469161 >> 172.21.15.122:0/4017469161 conn(0x7f8998079350 msgr2=0x7f899806b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:45.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 -- 172.21.15.122:0/4017469161 shutdown_connections 2024-05-03T17:55:45.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:45.155+0000 7f899e88e700 1 -- 172.21.15.122:0/4017469161 wait complete. 2024-05-03T17:55:45.887 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-03T17:55:46.128 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:46.580 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:46 smithi122 ceph-mon[29744]: pgmap v382: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:46.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:46 smithi149 ceph-mon[33409]: pgmap v382: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:46.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:46 smithi192 ceph-mon[33753]: pgmap v382: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.795+0000 7f8dd23f2700 1 -- 172.21.15.122:0/2043305613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dcc102680 msgr2=0x7f8dcc102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.795+0000 7f8dd23f2700 1 --2- 172.21.15.122:0/2043305613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dcc102680 0x7f8dcc102b00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009990 tx=0x7f8dc0009ca0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.796+0000 7f8dd23f2700 1 -- 172.21.15.122:0/2043305613 shutdown_connections 2024-05-03T17:55:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.796+0000 7f8dd23f2700 1 --2- 172.21.15.122:0/2043305613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dcc103040 0x7f8dcc10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.796+0000 7f8dd23f2700 1 --2- 172.21.15.122:0/2043305613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dcc107430 0x7f8dcc107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.796+0000 7f8dd23f2700 1 --2- 172.21.15.122:0/2043305613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dcc102680 0x7f8dcc102b00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.796+0000 7f8dd23f2700 1 -- 172.21.15.122:0/2043305613 >> 172.21.15.122:0/2043305613 conn(0x7f8dcc079290 msgr2=0x7f8dcc0796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.796+0000 7f8dd23f2700 1 -- 172.21.15.122:0/2043305613 shutdown_connections 2024-05-03T17:55:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.796+0000 7f8dd23f2700 1 -- 172.21.15.122:0/2043305613 wait complete. 2024-05-03T17:55:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.796+0000 7f8dd23f2700 1 Processor -- start 2024-05-03T17:55:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.796+0000 7f8dd23f2700 1 -- start start 2024-05-03T17:55:47.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd23f2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dcc102680 0x7f8dcc103ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd23f2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dcc103040 0x7f8dcc104420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd23f2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dcc107430 0x7f8dcc104960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd23f2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc112560 con 0x7f8dcc107430 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd23f2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc1123e0 con 0x7f8dcc102680 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd23f2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc1126e0 con 0x7f8dcc103040 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd098f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dcc107430 0x7f8dcc104960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dcb7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dcc103040 0x7f8dcc104420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd098f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dcc107430 0x7f8dcc104960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60310/0 (socket says 172.21.15.122:60310) 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd098f700 1 -- 172.21.15.122:0/3789337170 learned_addr learned my addr 172.21.15.122:0/3789337170 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dcbfff700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dcc102680 0x7f8dcc103ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd098f700 1 -- 172.21.15.122:0/3789337170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dcc103040 msgr2=0x7f8dcc104420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd098f700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dcc103040 0x7f8dcc104420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd098f700 1 -- 172.21.15.122:0/3789337170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dcc102680 msgr2=0x7f8dcc103ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd098f700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dcc102680 0x7f8dcc103ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd098f700 1 -- 172.21.15.122:0/3789337170 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc1a8040 con 0x7f8dcc107430 2024-05-03T17:55:47.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dcb7fe700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dcc103040 0x7f8dcc104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T17:55:47.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dd098f700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dcc107430 0x7f8dcc104960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400ada0 tx=0x7f8dc400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:47.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.797+0000 7f8dc97fa700 1 -- 172.21.15.122:0/3789337170 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4017020 con 0x7f8dcc107430 2024-05-03T17:55:47.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.798+0000 7f8dc97fa700 1 -- 172.21.15.122:0/3789337170 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dc400ce30 con 0x7f8dcc107430 2024-05-03T17:55:47.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.798+0000 7f8dd23f2700 1 -- 172.21.15.122:0/3789337170 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dcc1a8330 con 0x7f8dcc107430 2024-05-03T17:55:47.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.798+0000 7f8dd23f2700 1 -- 172.21.15.122:0/3789337170 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dcc1a87f0 con 0x7f8dcc107430 2024-05-03T17:55:47.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.798+0000 7f8dc97fa700 1 -- 172.21.15.122:0/3789337170 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4005350 con 0x7f8dcc107430 2024-05-03T17:55:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.800+0000 7f8dc97fa700 1 -- 172.21.15.122:0/3789337170 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8dc400c980 con 0x7f8dcc107430 2024-05-03T17:55:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.800+0000 7f8dc97fa700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8db40b2910 0x7f8db40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.801+0000 7f8dcbfff700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8db40b2910 0x7f8db40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.801+0000 7f8dc97fa700 1 -- 172.21.15.122:0/3789337170 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f8dc40e1a00 con 0x7f8dcc107430 2024-05-03T17:55:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.801+0000 7f8dcbfff700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8db40b2910 0x7f8db40b4dd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc005fd0 tx=0x7f8dbc00af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8ddcc85b80 1 Processor -- start 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8ddcc85b80 1 -- start start 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651278b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651273f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 0x5626512748c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5626511c68f0 con 0x562651279060 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56265112bd50 con 0x562651278740 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5626511c28b0 con 0x562651274480 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8dcb7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651273f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8dcb7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651273f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60314/0 (socket says 172.21.15.122:60314) 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8dcb7fe700 1 -- 172.21.15.122:0/1100254928 learned_addr learned my addr 172.21.15.122:0/1100254928 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8dcbfff700 1 --2- 172.21.15.122:0/1100254928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651278b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8dcb7fe700 1 -- 172.21.15.122:0/1100254928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 msgr2=0x5626512748c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8dcb7fe700 1 --2- 172.21.15.122:0/1100254928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 0x5626512748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8dcb7fe700 1 -- 172.21.15.122:0/1100254928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 msgr2=0x562651278b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8dcb7fe700 1 --2- 172.21.15.122:0/1100254928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651278b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8dcb7fe700 1 -- 172.21.15.122:0/1100254928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562651113880 con 0x562651279060 2024-05-03T17:55:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8dcb7fe700 1 --2- 172.21.15.122:0/1100254928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651273f40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009990 tx=0x7f8dc0003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33ad9708c4ac4c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.802+0000 7f8db27fc700 1 -- 172.21.15.122:0/1100254928 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0003860 con 0x562651279060 2024-05-03T17:55:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.803+0000 7f8db27fc700 1 -- 172.21.15.122:0/1100254928 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dc0026070 con 0x562651279060 2024-05-03T17:55:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.803+0000 7f8db27fc700 1 -- 172.21.15.122:0/1100254928 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc000f460 con 0x562651279060 2024-05-03T17:55:47.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.803+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1100254928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 msgr2=0x562651273f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:47.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.803+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/1100254928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651273f40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009990 tx=0x7f8dc0003730 comp rx=0 tx=0).stop 2024-05-03T17:55:47.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.803+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1100254928 shutdown_connections 2024-05-03T17:55:47.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.803+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/1100254928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 0x5626512748c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.803+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/1100254928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651278b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.803+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/1100254928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651273f40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.803+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1100254928 >> 172.21.15.122:0/1100254928 conn(0x5626511a0b20 msgr2=0x56265126e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.803+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1100254928 shutdown_connections 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.803+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1100254928 wait complete. 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8ddcc85b80 1 -- start start 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8ddcc85b80 1 Processor -- start 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651274480 0x56265127fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651278740 0x56265127d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651279060 0x56265127d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562651280b40 con 0x562651274480 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56265117fe80 con 0x562651279060 2024-05-03T17:55:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562651280cc0 con 0x562651278740 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8ddcc85b80 10 client.0 _ll_register_callbacks cb 0x56265115a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8dcbfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651274480 0x56265127fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8dcbfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651274480 0x56265127fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60324/0 (socket says 172.21.15.122:60324) 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.804+0000 7f8dcbfff700 1 -- 172.21.15.122:0/1243920340 learned_addr learned my addr 172.21.15.122:0/1243920340 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dd098f700 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651279060 0x56265127d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dcb7fe700 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651278740 0x56265127d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dcbfff700 1 -- 172.21.15.122:0/1243920340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651278740 msgr2=0x56265127d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dcbfff700 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651278740 0x56265127d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dcbfff700 1 -- 172.21.15.122:0/1243920340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651279060 msgr2=0x56265127d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dcbfff700 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651279060 0x56265127d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dcbfff700 1 -- 172.21.15.122:0/1243920340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56265117d710 con 0x562651274480 2024-05-03T17:55:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dcb7fe700 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651278740 0x56265127d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dd098f700 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651279060 0x56265127d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dcbfff700 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651274480 0x56265127fcc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0109e0 tx=0x7f8dbc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b114eb00e472911 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dab7fe700 1 -- 172.21.15.122:0/1243920340 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00fc60 con 0x562651274480 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dab7fe700 1 -- 172.21.15.122:0/1243920340 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f8dbc00fe00 con 0x562651274480 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1243920340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dc0026070 con 0x562651274480 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dab7fe700 1 -- 172.21.15.122:0/1243920340 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8dbc010e50 con 0x562651274480 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8ddcc85b80 10 client.15144 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1243920340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56265128b350 con 0x562651274480 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.805+0000 7f8dab7fe700 1 -- 172.21.15.122:0/1243920340 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dc0026070 con 0x562651274480 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.806+0000 7f8dab7fe700 1 -- 172.21.15.122:0/1243920340 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8dbc01d4e0 con 0x562651274480 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.806+0000 7f8ddcc85b80 10 client.15144 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.806+0000 7f8ddcc85b80 10 client.15144 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.806+0000 7f8ddcc85b80 20 client.15144 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.806+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x56265128da10 0x56265128ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.806+0000 7f8ddcc85b80 4 client.15144 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.806+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1243920340 --> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626511d42d0 con 0x56265128da10 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.806+0000 7f8dcb7fe700 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x56265128da10 0x56265128ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.806+0000 7f8dcb7fe700 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x56265128da10 0x56265128ee80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d800f748eeebd54 server_cookie=423f144eeda154bd in_seq=0 out_seq=0 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.806+0000 7f8dab7fe700 10 client.15144 ms_handle_connect on v2:172.21.15.192:6836/3476407415 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8dab7fe700 1 -- 172.21.15.122:0/1243920340 <== mds.1 v2:172.21.15.192:6836/3476407415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5626511d42d0 con 0x56265128da10 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8dab7fe700 10 client.15144 handle_command_reply: tid=0 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8ddcc85b80 1 client.15144 shutdown 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1243920340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651274480 msgr2=0x56265127fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651274480 0x56265127fcc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0109e0 tx=0x7f8dbc0047c0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1243920340 shutdown_connections 2024-05-03T17:55:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x56265128da10 0x56265128ee80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651279060 0x56265127d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8dab7fe700 0 client.15144 ms_handle_reset on v2:172.21.15.192:6836/3476407415 2024-05-03T17:55:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651274480 0x56265127fcc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/1243920340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651278740 0x56265127d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1243920340 >> 172.21.15.122:0/1243920340 conn(0x5626511a0b20 msgr2=0x562651279d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1243920340 shutdown_connections 2024-05-03T17:55:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.807+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/1243920340 wait complete. 2024-05-03T17:55:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.808+0000 7f8ddcc85b80 20 client.15144 trim_cache size 0 max 16384 2024-05-03T17:55:47.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8ddcc85b80 1 Processor -- start 2024-05-03T17:55:47.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8ddcc85b80 1 -- start start 2024-05-03T17:55:47.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 0x56265126e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651271f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651272470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56265128d1c0 con 0x562651279060 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56265128d890 con 0x562651278740 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56265128df60 con 0x562651274480 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8dd098f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651272470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8dd098f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651272470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60330/0 (socket says 172.21.15.122:60330) 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8dd098f700 1 -- 172.21.15.122:0/3676880214 learned_addr learned my addr 172.21.15.122:0/3676880214 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8dcb7fe700 1 --2- 172.21.15.122:0/3676880214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651271f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8dcbfff700 1 --2- 172.21.15.122:0/3676880214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 0x56265126e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8dd098f700 1 -- 172.21.15.122:0/3676880214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 msgr2=0x56265126e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8dd098f700 1 --2- 172.21.15.122:0/3676880214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 0x56265126e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8dd098f700 1 -- 172.21.15.122:0/3676880214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 msgr2=0x562651271f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8dd098f700 1 --2- 172.21.15.122:0/3676880214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651271f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.828+0000 7f8dd098f700 1 -- 172.21.15.122:0/3676880214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56265126f5d0 con 0x562651279060 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8dd098f700 1 --2- 172.21.15.122:0/3676880214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651272470 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40a50d0 tx=0x7f8dc400cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcde7846afde8c92 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8db1ffb700 1 -- 172.21.15.122:0/3676880214 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40a9050 con 0x562651279060 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8db1ffb700 1 -- 172.21.15.122:0/3676880214 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dc40ad400 con 0x562651279060 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8db1ffb700 1 -- 172.21.15.122:0/3676880214 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40b1990 con 0x562651279060 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/3676880214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 msgr2=0x562651272470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/3676880214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651272470 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40a50d0 tx=0x7f8dc400cd40 comp rx=0 tx=0).stop 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/3676880214 shutdown_connections 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/3676880214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651272470 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/3676880214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 0x56265126e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/3676880214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651271f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/3676880214 >> 172.21.15.122:0/3676880214 conn(0x5626511a0b20 msgr2=0x56265127b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:47.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/3676880214 shutdown_connections 2024-05-03T17:55:47.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.829+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/3676880214 wait complete. 2024-05-03T17:55:47.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8ddcc85b80 1 -- start start 2024-05-03T17:55:47.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8ddcc85b80 1 Processor -- start 2024-05-03T17:55:47.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 0x5626512982f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651298830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8ddcc85b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651271b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56265128ed00 con 0x562651279060 2024-05-03T17:55:47.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56265128e630 con 0x562651278740 2024-05-03T17:55:47.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8ddcc85b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56265128f3d0 con 0x562651274480 2024-05-03T17:55:47.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8ddcc85b80 10 client.0 _ll_register_callbacks cb 0x56265115a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:47.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8dd098f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651271b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8dd098f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651271b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60332/0 (socket says 172.21.15.122:60332) 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.830+0000 7f8dd098f700 1 -- 172.21.15.122:0/940453724 learned_addr learned my addr 172.21.15.122:0/940453724 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dcb7fe700 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651298830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dd098f700 1 -- 172.21.15.122:0/940453724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 msgr2=0x5626512982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dd098f700 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 0x5626512982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dd098f700 1 -- 172.21.15.122:0/940453724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 msgr2=0x562651298830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dd098f700 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651298830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dd098f700 1 -- 172.21.15.122:0/940453724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626512720b0 con 0x562651279060 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dd098f700 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651271b70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40a8fe0 tx=0x7f8dc4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9263319e1312935f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dab7fe700 1 -- 172.21.15.122:0/940453724 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc406c020 con 0x562651279060 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dab7fe700 1 -- 172.21.15.122:0/940453724 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f8dc40a5640 con 0x562651279060 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/940453724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dc40ad400 con 0x562651279060 2024-05-03T17:55:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dab7fe700 1 -- 172.21.15.122:0/940453724 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8dc40b1ba0 con 0x562651279060 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8ddcc85b80 10 client.15150 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/940453724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5626512f9600 con 0x562651279060 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dab7fe700 1 -- 172.21.15.122:0/940453724 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dc40ad400 con 0x562651279060 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8dab7fe700 1 -- 172.21.15.122:0/940453724 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8dc4012070 con 0x562651279060 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8ddcc85b80 10 client.15150 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.831+0000 7f8ddcc85b80 10 client.15150 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.832+0000 7f8ddcc85b80 20 client.15150 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.832+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x5626512fdd00 0x5626512fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.832+0000 7f8ddcc85b80 4 client.15150 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.832+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/940453724 --> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] -- command(tid 0: {"prefix": "status"}) -- 0x5626511ce370 con 0x5626512fdd00 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.832+0000 7f8dcbfff700 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x5626512fdd00 0x5626512fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.832+0000 7f8dab7fe700 10 client.15150 ms_handle_connect on v2:172.21.15.192:6836/3476407415 2024-05-03T17:55:47.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.832+0000 7f8dcbfff700 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x5626512fdd00 0x5626512fca10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdf159b0e8ad56c8 server_cookie=24419a4a0ceb1fe1 in_seq=0 out_seq=0 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8dab7fe700 1 -- 172.21.15.122:0/940453724 <== mds.1 v2:172.21.15.192:6836/3476407415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5626511ce370 con 0x5626512fdd00 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8dab7fe700 10 client.15150 handle_command_reply: tid=0 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 1 client.15150 shutdown 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/940453724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 msgr2=0x562651271b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651271b70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40a8fe0 tx=0x7f8dc4007a00 comp rx=0 tx=0).stop 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/940453724 shutdown_connections 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.192:6836/3476407415,v1:172.21.15.192:6837/3476407415] conn(0x5626512fdd00 0x5626512fca10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562651279060 0x562651271b70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562651274480 0x5626512982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 1 --2- 172.21.15.122:0/940453724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562651278740 0x562651298830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/940453724 >> 172.21.15.122:0/940453724 conn(0x5626511a0b20 msgr2=0x56265127be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8dab7fe700 0 client.15150 ms_handle_reset on v2:172.21.15.192:6836/3476407415 2024-05-03T17:55:47.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/940453724 shutdown_connections 2024-05-03T17:55:47.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 1 -- 172.21.15.122:0/940453724 wait complete. 2024-05-03T17:55:47.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.833+0000 7f8ddcc85b80 20 client.15150 trim_cache size 0 max 16384 2024-05-03T17:55:47.835 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-05-03T17:55:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "a46fc846-0974-11ef-bc97-c7b262605968", 2024-05-03T17:55:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 1, 2024-05-03T17:55:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24463, 2024-05-03T17:55:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-05-03T17:55:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-05-03T17:55:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-05-03T17:55:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 178.06388554099999, 2024-05-03T17:55:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 24, 2024-05-03T17:55:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 79, 2024-05-03T17:55:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 77, 2024-05-03T17:55:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 222.90644215699999 2024-05-03T17:55:47.836 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 -- 172.21.15.122:0/3789337170 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8db40b2910 msgr2=0x7f8db40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8db40b2910 0x7f8db40b4dd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc005fd0 tx=0x7f8dbc00af20 comp rx=0 tx=0).stop 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 -- 172.21.15.122:0/3789337170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dcc107430 msgr2=0x7f8dcc104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dcc107430 0x7f8dcc104960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400ada0 tx=0x7f8dc400c330 comp rx=0 tx=0).stop 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 -- 172.21.15.122:0/3789337170 shutdown_connections 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f8db40b2910 0x7f8db40b4dd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dcc107430 0x7f8dcc104960 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dcc102680 0x7f8dcc103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 --2- 172.21.15.122:0/3789337170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dcc103040 0x7f8dcc104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 -- 172.21.15.122:0/3789337170 >> 172.21.15.122:0/3789337170 conn(0x7f8dcc079290 msgr2=0x7f8dcc06bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 -- 172.21.15.122:0/3789337170 shutdown_connections 2024-05-03T17:55:47.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:47.834+0000 7f8dd23f2700 1 -- 172.21.15.122:0/3789337170 wait complete. 2024-05-03T17:55:48.611 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:48 smithi122 ceph-mon[29744]: pgmap v383: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:48.638 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-03T17:55:48.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:48 smithi149 ceph-mon[33409]: pgmap v383: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:48.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:48 smithi192 ceph-mon[33753]: pgmap v383: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:49.686 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:49 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:55:49.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:49 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:55:49.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:49 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:55:50.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.552+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/4014011242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f406c3d0 msgr2=0x7fd0f4110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 --2- 172.21.15.122:0/4014011242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f406c3d0 0x7fd0f4110320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00b0a0 tx=0x7fd0ec00b3b0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/4014011242 shutdown_connections 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 --2- 172.21.15.122:0/4014011242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f406c3d0 0x7fd0f4110320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 --2- 172.21.15.122:0/4014011242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0f4109500 0x7fd0f41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 --2- 172.21.15.122:0/4014011242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f406ba10 0x7fd0f406be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/4014011242 >> 172.21.15.122:0/4014011242 conn(0x7fd0f4079310 msgr2=0x7fd0f4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/4014011242 shutdown_connections 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/4014011242 wait complete. 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 Processor -- start 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 -- start start 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f406ba10 0x7fd0f419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0f406c3d0 0x7fd0f419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.553+0000 7fd0fc0bf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f4109500 0x7fd0f4199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0fc0bf700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f4113630 con 0x7fd0f406ba10 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0fc0bf700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f41134b0 con 0x7fd0f406c3d0 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0fc0bf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f41137b0 con 0x7fd0f4109500 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0f9e5b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f406ba10 0x7fd0f419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0f9e5b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f406ba10 0x7fd0f419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45166/0 (socket says 172.21.15.122:45166) 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0f9e5b700 1 -- 172.21.15.122:0/3208849419 learned_addr learned my addr 172.21.15.122:0/3208849419 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0fa65c700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f4109500 0x7fd0f4199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0f965a700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0f406c3d0 0x7fd0f419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0f9e5b700 1 -- 172.21.15.122:0/3208849419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f4109500 msgr2=0x7fd0f4199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0f9e5b700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f4109500 0x7fd0f4199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0f9e5b700 1 -- 172.21.15.122:0/3208849419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0f406c3d0 msgr2=0x7fd0f419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0f9e5b700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0f406c3d0 0x7fd0f419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0f9e5b700 1 -- 172.21.15.122:0/3208849419 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f419a070 con 0x7fd0f406ba10 2024-05-03T17:55:50.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0f9e5b700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f406ba10 0x7fd0f419f750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000d980 tx=0x7fd0f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:50.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0eaffd700 1 -- 172.21.15.122:0/3208849419 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0016070 con 0x7fd0f406ba10 2024-05-03T17:55:50.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/3208849419 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f419a360 con 0x7fd0f406ba10 2024-05-03T17:55:50.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/3208849419 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f41a6620 con 0x7fd0f406ba10 2024-05-03T17:55:50.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0eaffd700 1 -- 172.21.15.122:0/3208849419 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f0004830 con 0x7fd0f406ba10 2024-05-03T17:55:50.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.554+0000 7fd0eaffd700 1 -- 172.21.15.122:0/3208849419 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00054c0 con 0x7fd0f406ba10 2024-05-03T17:55:50.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.557+0000 7fd0eaffd700 1 -- 172.21.15.122:0/3208849419 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fd0f0004220 con 0x7fd0f406ba10 2024-05-03T17:55:50.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.557+0000 7fd0eaffd700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd0e00b2990 0x7fd0e00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.557+0000 7fd0eaffd700 1 -- 172.21.15.122:0/3208849419 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fd0f00e1f40 con 0x7fd0f406ba10 2024-05-03T17:55:50.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.557+0000 7fd0f965a700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd0e00b2990 0x7fd0e00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.558+0000 7fd0f965a700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd0e00b2990 0x7fd0e00b4e50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005950 tx=0x7fd0e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:50.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.558+0000 7fd106952b80 1 Processor -- start 2024-05-03T17:55:50.553 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:50 smithi122 ceph-mon[29744]: pgmap v384: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:50.553 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:50.554 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:50.554 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:50.554 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:50.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.558+0000 7fd106952b80 1 -- start start 2024-05-03T17:55:50.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.558+0000 7fd106952b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957c530 0x56463957c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.558+0000 7fd106952b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 0x564639580890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.558+0000 7fd106952b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 0x564639581210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.558+0000 7fd106952b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5646394ca8f0 con 0x564639580dd0 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.558+0000 7fd106952b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56463942fd50 con 0x56463957c530 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.558+0000 7fd106952b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5646394c68b0 con 0x56463957ce50 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0fa65c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 0x564639581210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0fa65c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 0x564639581210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45178/0 (socket says 172.21.15.122:45178) 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0fa65c700 1 -- 172.21.15.122:0/2448893852 learned_addr learned my addr 172.21.15.122:0/2448893852 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0f9e5b700 1 --2- 172.21.15.122:0/2448893852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957c530 0x56463957c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0f965a700 1 --2- 172.21.15.122:0/2448893852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 0x564639580890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0fa65c700 1 -- 172.21.15.122:0/2448893852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 msgr2=0x564639580890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0fa65c700 1 --2- 172.21.15.122:0/2448893852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 0x564639580890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0fa65c700 1 -- 172.21.15.122:0/2448893852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957c530 msgr2=0x56463957c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0fa65c700 1 --2- 172.21.15.122:0/2448893852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957c530 0x56463957c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0fa65c700 1 -- 172.21.15.122:0/2448893852 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564639417880 con 0x564639580dd0 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0fa65c700 1 --2- 172.21.15.122:0/2448893852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 0x564639581210 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009100 tx=0x7fd0ec009130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4073b349d66b6b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.559+0000 7fd0dffff700 1 -- 172.21.15.122:0/2448893852 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec020070 con 0x564639580dd0 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.560+0000 7fd0dffff700 1 -- 172.21.15.122:0/2448893852 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ec0074e0 con 0x564639580dd0 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.560+0000 7fd0dffff700 1 -- 172.21.15.122:0/2448893852 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec005280 con 0x564639580dd0 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.560+0000 7fd106952b80 1 -- 172.21.15.122:0/2448893852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 msgr2=0x564639581210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.560+0000 7fd106952b80 1 --2- 172.21.15.122:0/2448893852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 0x564639581210 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009100 tx=0x7fd0ec009130 comp rx=0 tx=0).stop 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.560+0000 7fd106952b80 1 -- 172.21.15.122:0/2448893852 shutdown_connections 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.560+0000 7fd106952b80 1 --2- 172.21.15.122:0/2448893852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 0x564639581210 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.560+0000 7fd106952b80 1 --2- 172.21.15.122:0/2448893852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957c530 0x56463957c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.560+0000 7fd106952b80 1 --2- 172.21.15.122:0/2448893852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 0x564639580890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.560+0000 7fd106952b80 1 -- 172.21.15.122:0/2448893852 >> 172.21.15.122:0/2448893852 conn(0x5646394a4b20 msgr2=0x5646395729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:50.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.560+0000 7fd106952b80 1 -- 172.21.15.122:0/2448893852 shutdown_connections 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.560+0000 7fd106952b80 1 -- 172.21.15.122:0/2448893852 wait complete. 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.561+0000 7fd106952b80 1 -- start start 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.561+0000 7fd106952b80 1 Processor -- start 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.561+0000 7fd106952b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957c530 0x564639588220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.561+0000 7fd106952b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 0x5646395855c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.561+0000 7fd106952b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564639580dd0 0x564639585b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.561+0000 7fd106952b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564639588ee0 con 0x56463957c530 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.561+0000 7fd106952b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564639483e80 con 0x564639580dd0 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.561+0000 7fd106952b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564639589060 con 0x56463957ce50 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd106952b80 10 client.0 _ll_register_callbacks cb 0x56463945e560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0f965a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 0x5646395855c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0f965a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 0x5646395855c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.122:38530/0 (socket says 172.21.15.122:38530) 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0f9e5b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957c530 0x564639588220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0f965a700 1 -- 172.21.15.122:0/3185623429 learned_addr learned my addr 172.21.15.122:0/3185623429 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:50.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0fa65c700 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564639580dd0 0x564639585b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0f965a700 1 -- 172.21.15.122:0/3185623429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564639580dd0 msgr2=0x564639585b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:50.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0f965a700 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564639580dd0 0x564639585b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0f965a700 1 -- 172.21.15.122:0/3185623429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957c530 msgr2=0x564639588220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:50.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0f965a700 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957c530 0x564639588220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0f965a700 1 -- 172.21.15.122:0/3185623429 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564639481710 con 0x56463957ce50 2024-05-03T17:55:50.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0fa65c700 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564639580dd0 0x564639585b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:50.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0f965a700 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 0x5646395855c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400fea0 tx=0x7fd0e4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75133361aad97f04 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/3185623429 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4028670 con 0x56463957ce50 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/3185623429 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fd0e4028810 con 0x56463957ce50 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.562+0000 7fd106952b80 1 -- 172.21.15.122:0/3185623429 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ec0074e0 con 0x56463957ce50 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/3185623429 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd0e4028a10 con 0x56463957ce50 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd106952b80 10 client.24838 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd106952b80 1 -- 172.21.15.122:0/3185623429 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564639593820 con 0x56463957ce50 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/3185623429 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ec0074e0 con 0x56463957ce50 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/3185623429 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd0e401e430 con 0x56463957ce50 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd106952b80 10 client.24838 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd106952b80 10 client.24838 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd106952b80 20 client.24838 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd106952b80 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x564639595f10 0x564639597380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd106952b80 4 client.24838 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd106952b80 1 -- 172.21.15.122:0/3185623429 --> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646394d82d0 con 0x564639595f10 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.563+0000 7fd0f9e5b700 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x564639595f10 0x564639597380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.564+0000 7fd0dd7fa700 10 client.24838 ms_handle_connect on v2:172.21.15.192:6838/1429818440 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.564+0000 7fd0f9e5b700 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x564639595f10 0x564639597380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db1bad127f9a595 server_cookie=d442ff85242255a9 in_seq=0 out_seq=0 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.564+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/3185623429 <== mds.0 v2:172.21.15.192:6838/1429818440 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646394d82d0 con 0x564639595f10 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.564+0000 7fd0dd7fa700 10 client.24838 handle_command_reply: tid=0 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.564+0000 7fd106952b80 1 client.24838 shutdown 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd106952b80 1 -- 172.21.15.122:0/3185623429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 msgr2=0x5646395855c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd106952b80 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 0x5646395855c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400fea0 tx=0x7fd0e4004750 comp rx=0 tx=0).stop 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd106952b80 1 -- 172.21.15.122:0/3185623429 shutdown_connections 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd106952b80 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564639580dd0 0x564639585b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd106952b80 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x564639595f10 0x564639597380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd106952b80 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957c530 0x564639588220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd106952b80 1 --2- 172.21.15.122:0/3185623429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957ce50 0x5646395855c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd0dd7fa700 0 client.24838 ms_handle_reset on v2:172.21.15.192:6838/1429818440 2024-05-03T17:55:50.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd106952b80 1 -- 172.21.15.122:0/3185623429 >> 172.21.15.122:0/3185623429 conn(0x5646394a4b20 msgr2=0x564639582110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:50.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd106952b80 1 -- 172.21.15.122:0/3185623429 shutdown_connections 2024-05-03T17:55:50.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd106952b80 1 -- 172.21.15.122:0/3185623429 wait complete. 2024-05-03T17:55:50.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.565+0000 7fd106952b80 20 client.24838 trim_cache size 0 max 16384 2024-05-03T17:55:50.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.587+0000 7fd106952b80 1 Processor -- start 2024-05-03T17:55:50.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.587+0000 7fd106952b80 1 -- start start 2024-05-03T17:55:50.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.587+0000 7fd106952b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957c530 0x5646395811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.587+0000 7fd106952b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957ce50 0x564639584b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.587+0000 7fd106952b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564639580dd0 0x564639585580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.587+0000 7fd106952b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5646395956c0 con 0x56463957ce50 2024-05-03T17:55:50.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.587+0000 7fd106952b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564639595d90 con 0x56463957c530 2024-05-03T17:55:50.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.587+0000 7fd106952b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564639596460 con 0x564639580dd0 2024-05-03T17:55:50.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0f965a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957ce50 0x564639584b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0f965a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957ce50 0x564639584b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45196/0 (socket says 172.21.15.122:45196) 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0f965a700 1 -- 172.21.15.122:0/4240079627 learned_addr learned my addr 172.21.15.122:0/4240079627 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0f9e5b700 1 --2- 172.21.15.122:0/4240079627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957c530 0x5646395811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0fa65c700 1 --2- 172.21.15.122:0/4240079627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564639580dd0 0x564639585580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0f965a700 1 -- 172.21.15.122:0/4240079627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564639580dd0 msgr2=0x564639585580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0f965a700 1 --2- 172.21.15.122:0/4240079627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564639580dd0 0x564639585580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0f965a700 1 -- 172.21.15.122:0/4240079627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957c530 msgr2=0x5646395811b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0f965a700 1 --2- 172.21.15.122:0/4240079627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957c530 0x5646395811b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0f965a700 1 -- 172.21.15.122:0/4240079627 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564639573600 con 0x56463957ce50 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0f965a700 1 --2- 172.21.15.122:0/4240079627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957ce50 0x564639584b20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400ba40 tx=0x7fd0e4005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebdc65d5f5e86e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0df7fe700 1 -- 172.21.15.122:0/4240079627 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e401e630 con 0x56463957ce50 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0df7fe700 1 -- 172.21.15.122:0/4240079627 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0e4023070 con 0x56463957ce50 2024-05-03T17:55:50.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.588+0000 7fd0df7fe700 1 -- 172.21.15.122:0/4240079627 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e401edf0 con 0x56463957ce50 2024-05-03T17:55:50.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.589+0000 7fd106952b80 1 -- 172.21.15.122:0/4240079627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957ce50 msgr2=0x564639584b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:50.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.589+0000 7fd106952b80 1 --2- 172.21.15.122:0/4240079627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957ce50 0x564639584b20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400ba40 tx=0x7fd0e4005d60 comp rx=0 tx=0).stop 2024-05-03T17:55:50.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.589+0000 7fd106952b80 1 -- 172.21.15.122:0/4240079627 shutdown_connections 2024-05-03T17:55:50.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.589+0000 7fd106952b80 1 --2- 172.21.15.122:0/4240079627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564639580dd0 0x564639585580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.589+0000 7fd106952b80 1 --2- 172.21.15.122:0/4240079627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957c530 0x5646395811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.589+0000 7fd106952b80 1 --2- 172.21.15.122:0/4240079627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463957ce50 0x564639584b20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.589+0000 7fd106952b80 1 -- 172.21.15.122:0/4240079627 >> 172.21.15.122:0/4240079627 conn(0x5646394a4b20 msgr2=0x5646395728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:50.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.589+0000 7fd106952b80 1 -- 172.21.15.122:0/4240079627 shutdown_connections 2024-05-03T17:55:50.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.589+0000 7fd106952b80 1 -- 172.21.15.122:0/4240079627 wait complete. 2024-05-03T17:55:50.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd106952b80 1 -- start start 2024-05-03T17:55:50.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd106952b80 1 Processor -- start 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd106952b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957c530 0x56463957f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd106952b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957ce50 0x5646395a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd106952b80 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 0x56463957fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd106952b80 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564639597200 con 0x564639580dd0 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd106952b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564639596b30 con 0x56463957ce50 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd106952b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5646395978d0 con 0x56463957c530 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd106952b80 10 client.0 _ll_register_callbacks cb 0x56463945e560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd0f965a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957ce50 0x5646395a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd0f965a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957ce50 0x5646395a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.122:39150/0 (socket says 172.21.15.122:39150) 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd0f965a700 1 -- 172.21.15.122:0/1352791515 learned_addr learned my addr 172.21.15.122:0/1352791515 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd0f9e5b700 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957c530 0x56463957f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.590+0000 7fd0fa65c700 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 0x56463957fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd0f965a700 1 -- 172.21.15.122:0/1352791515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957c530 msgr2=0x56463957f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd0f965a700 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957c530 0x56463957f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd0f965a700 1 -- 172.21.15.122:0/1352791515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 msgr2=0x56463957fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd0f965a700 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 0x56463957fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd0f965a700 1 -- 172.21.15.122:0/1352791515 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646395a1230 con 0x56463957ce50 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd0f965a700 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957ce50 0x5646395a07f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400fea0 tx=0x7fd0e4028de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a2615316bdbc777 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd0dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/1352791515 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e401ea70 con 0x56463957ce50 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd106952b80 1 -- 172.21.15.122:0/1352791515 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0e4023070 con 0x56463957ce50 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/1352791515 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fd0e401ec10 con 0x56463957ce50 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/1352791515 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd0e401ee10 con 0x56463957ce50 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd106952b80 10 client.24800 fetch_fsmap learned FSMap version 24 2024-05-03T17:55:50.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd106952b80 1 -- 172.21.15.122:0/1352791515 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5646396018f0 con 0x56463957ce50 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.591+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/1352791515 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0e4023070 con 0x56463957ce50 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.592+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/1352791515 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd0e4004980 con 0x56463957ce50 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.592+0000 7fd106952b80 10 client.24800 fetch_fsmap finished waiting for FSMap version 24 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.592+0000 7fd106952b80 10 client.24800 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.592+0000 7fd106952b80 20 client.24800 populate_metadata read hostname 'smithi122' 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.592+0000 7fd106952b80 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x564639606200 0x564639604f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.592+0000 7fd106952b80 4 client.24800 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.592+0000 7fd106952b80 1 -- 172.21.15.122:0/1352791515 --> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] -- command(tid 0: {"prefix": "status"}) -- 0x5646394d2370 con 0x564639606200 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.592+0000 7fd0f9e5b700 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x564639606200 0x564639604f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.592+0000 7fd0dd7fa700 10 client.24800 ms_handle_connect on v2:172.21.15.192:6838/1429818440 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.592+0000 7fd0f9e5b700 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x564639606200 0x564639604f10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9730acbe9c79d131 server_cookie=bc842a081fb94e85 in_seq=0 out_seq=0 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd0dd7fa700 1 -- 172.21.15.122:0/1352791515 <== mds.0 v2:172.21.15.192:6838/1429818440 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5646394d2370 con 0x564639606200 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd0dd7fa700 10 client.24800 handle_command_reply: tid=0 2024-05-03T17:55:50.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 1 client.24800 shutdown 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 1 -- 172.21.15.122:0/1352791515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957ce50 msgr2=0x5646395a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957ce50 0x5646395a07f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400fea0 tx=0x7fd0e4028de0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 1 -- 172.21.15.122:0/1352791515 shutdown_connections 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.192:6838/1429818440,v1:172.21.15.192:6839/1429818440] conn(0x564639606200 0x564639604f10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564639580dd0 0x56463957fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56463957c530 0x56463957f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd0dd7fa700 0 client.24800 ms_handle_reset on v2:172.21.15.192:6838/1429818440 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 1 --2- 172.21.15.122:0/1352791515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56463957ce50 0x5646395a07f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 1 -- 172.21.15.122:0/1352791515 >> 172.21.15.122:0/1352791515 conn(0x5646394a4b20 msgr2=0x564639584b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 1 -- 172.21.15.122:0/1352791515 shutdown_connections 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 1 -- 172.21.15.122:0/1352791515 wait complete. 2024-05-03T17:55:50.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd106952b80 20 client.24800 trim_cache size 0 max 16384 2024-05-03T17:55:50.594 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-05-03T17:55:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "a46fc846-0974-11ef-bc97-c7b262605968", 2024-05-03T17:55:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-05-03T17:55:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24481, 2024-05-03T17:55:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-05-03T17:55:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-05-03T17:55:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-05-03T17:55:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 192.880141213, 2024-05-03T17:55:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 24, 2024-05-03T17:55:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 79, 2024-05-03T17:55:50.594 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 77, 2024-05-03T17:55:50.595 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 220.00020948900001 2024-05-03T17:55:50.595 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-05-03T17:55:50.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/3208849419 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd0e00b2990 msgr2=0x7fd0e00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:50.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd0fc0bf700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd0e00b2990 0x7fd0e00b4e50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005950 tx=0x7fd0e40058e0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/3208849419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f406ba10 msgr2=0x7fd0f419f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:50.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.593+0000 7fd0fc0bf700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f406ba10 0x7fd0f419f750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000d980 tx=0x7fd0f000dc90 comp rx=0 tx=0).stop 2024-05-03T17:55:50.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.594+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/3208849419 shutdown_connections 2024-05-03T17:55:50.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.594+0000 7fd0fc0bf700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fd0e00b2990 0x7fd0e00b4e50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.594+0000 7fd0fc0bf700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f4109500 0x7fd0f4199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.594+0000 7fd0fc0bf700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f406ba10 0x7fd0f419f750 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.594+0000 7fd0fc0bf700 1 --2- 172.21.15.122:0/3208849419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0f406c3d0 0x7fd0f419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:50.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.594+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/3208849419 >> 172.21.15.122:0/3208849419 conn(0x7fd0f4079310 msgr2=0x7fd0f4100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:50.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.594+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/3208849419 shutdown_connections 2024-05-03T17:55:50.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:50.594+0000 7fd0fc0bf700 1 -- 172.21.15.122:0/3208849419 wait complete. 2024-05-03T17:55:50.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:50 smithi149 ceph-mon[33409]: pgmap v384: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:50.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:50 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:50.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:50 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:50.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:50 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:50.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:50 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:50.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:50 smithi192 ceph-mon[33753]: pgmap v384: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:55:50.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:50 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:50.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:50 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:50.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:50 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:50.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:50 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:51.266 INFO:teuthology.run_tasks:Running task sequential... 2024-05-03T17:55:51.275 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-03T17:55:51.275 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-03T17:55:51.276 INFO:teuthology.task.print:Enabling mgr modules 2024-05-03T17:55:51.276 INFO:teuthology.run_tasks:Running task kclient... 2024-05-03T17:55:51.304 INFO:tasks.kclient:Mounting kernel clients... 2024-05-03T17:55:51.304 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-03T17:55:51.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T17:55:51.304 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-05-03T17:55:51.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T17:55:51.321 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-05-03T17:55:51.387 INFO:teuthology.orchestra.run.smithi122.stderr:Cannot find device "ceph-brx" 2024-05-03T17:55:51.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T17:55:51.389 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-03T17:55:51.389 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-03T17:55:51.389 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-03T17:55:51.389 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-03T17:55:51.664 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T17:55:51.785 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:51 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:55:51.786 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:51 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:55:51.786 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:51 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:51.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:51 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:55:51.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:51 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:55:51.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:51 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:51.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:51 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:55:51.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:51 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:55:51.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:51 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:55:52.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:52 smithi149 ceph-mon[33409]: pgmap v385: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:52.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:52 smithi122 ceph-mon[29744]: pgmap v385: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:52.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:52 smithi192 ceph-mon[33753]: pgmap v385: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:55:53.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.310+0000 7f29f1315700 1 -- 172.21.15.122:0/2082515216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29ec1037b0 msgr2=0x7f29ec107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:53.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.310+0000 7f29f1315700 1 --2- 172.21.15.122:0/2082515216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29ec1037b0 0x7f29ec107030 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f29d40099c0 tx=0x7f29d4009cd0 comp rx=0 tx=0).stop 2024-05-03T17:55:53.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.310+0000 7f29f1315700 1 -- 172.21.15.122:0/2082515216 shutdown_connections 2024-05-03T17:55:53.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.310+0000 7f29f1315700 1 --2- 172.21.15.122:0/2082515216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ec107760 0x7f29ec10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:53.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.310+0000 7f29f1315700 1 --2- 172.21.15.122:0/2082515216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29ec102e00 0x7f29ec1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:53.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.310+0000 7f29f1315700 1 --2- 172.21.15.122:0/2082515216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29ec1037b0 0x7f29ec107030 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:53.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.310+0000 7f29f1315700 1 -- 172.21.15.122:0/2082515216 >> 172.21.15.122:0/2082515216 conn(0x7f29ec078a10 msgr2=0x7f29ec0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:53.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.310+0000 7f29f1315700 1 -- 172.21.15.122:0/2082515216 shutdown_connections 2024-05-03T17:55:53.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.310+0000 7f29f1315700 1 -- 172.21.15.122:0/2082515216 wait complete. 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.311+0000 7f29f1315700 1 Processor -- start 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.311+0000 7f29f1315700 1 -- start start 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29f1315700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29ec102e00 0x7f29ec19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29f1315700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ec1037b0 0x7f29ec19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29f1315700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29ec107760 0x7f29ec1a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29f1315700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec112540 con 0x7f29ec107760 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29f1315700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec1123c0 con 0x7f29ec102e00 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29f1315700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec1126c0 con 0x7f29ec1037b0 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29eb7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29ec107760 0x7f29ec1a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29eb7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29ec107760 0x7f29ec1a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45230/0 (socket says 172.21.15.122:45230) 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29eb7fe700 1 -- 172.21.15.122:0/2685489666 learned_addr learned my addr 172.21.15.122:0/2685489666 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29e27fc700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ec1037b0 0x7f29ec19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:53.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29eaffd700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29ec102e00 0x7f29ec19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:53.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29eb7fe700 1 -- 172.21.15.122:0/2685489666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ec1037b0 msgr2=0x7f29ec19aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:53.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29eb7fe700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ec1037b0 0x7f29ec19aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:53.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29eb7fe700 1 -- 172.21.15.122:0/2685489666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29ec102e00 msgr2=0x7f29ec19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:53.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29eb7fe700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29ec102e00 0x7f29ec19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:53.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29eb7fe700 1 -- 172.21.15.122:0/2685489666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ec1a5f20 con 0x7f29ec107760 2024-05-03T17:55:53.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29eb7fe700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29ec107760 0x7f29ec1a59e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f29e4007ee0 tx=0x7f29e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:53.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29e8ff9700 1 -- 172.21.15.122:0/2685489666 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4004bb0 con 0x7f29ec107760 2024-05-03T17:55:53.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.312+0000 7f29f1315700 1 -- 172.21.15.122:0/2685489666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ec1a60f0 con 0x7f29ec107760 2024-05-03T17:55:53.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.313+0000 7f29e8ff9700 1 -- 172.21.15.122:0/2685489666 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e4004d50 con 0x7f29ec107760 2024-05-03T17:55:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.313+0000 7f29e8ff9700 1 -- 172.21.15.122:0/2685489666 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4005920 con 0x7f29ec107760 2024-05-03T17:55:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.313+0000 7f29f1315700 1 -- 172.21.15.122:0/2685489666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ec1a6600 con 0x7f29ec107760 2024-05-03T17:55:53.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.314+0000 7f29f1315700 1 -- 172.21.15.122:0/2685489666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29ec06a190 con 0x7f29ec107760 2024-05-03T17:55:53.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.315+0000 7f29e8ff9700 1 -- 172.21.15.122:0/2685489666 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f29e401a020 con 0x7f29ec107760 2024-05-03T17:55:53.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.316+0000 7f29e8ff9700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29d80b2990 0x7f29d80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:53.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.316+0000 7f29eaffd700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29d80b2990 0x7f29d80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:53.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.316+0000 7f29e8ff9700 1 -- 172.21.15.122:0/2685489666 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f29e40e1850 con 0x7f29ec107760 2024-05-03T17:55:53.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.316+0000 7f29eaffd700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29d80b2990 0x7f29d80b4e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f29dc005950 tx=0x7f29dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.320+0000 7f29e8ff9700 1 -- 172.21.15.122:0/2685489666 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f29e4010070 con 0x7f29ec107760 2024-05-03T17:55:53.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.571+0000 7f29f1315700 1 -- 172.21.15.122:0/2685489666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f29ec02e510 con 0x7f29ec107760 2024-05-03T17:55:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.572+0000 7f29e8ff9700 1 -- 172.21.15.122:0/2685489666 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f29e40a9510 con 0x7f29ec107760 2024-05-03T17:55:53.567 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-03T17:55:53.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 -- 172.21.15.122:0/2685489666 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29d80b2990 msgr2=0x7f29d80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:53.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29d80b2990 0x7f29d80b4e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f29dc005950 tx=0x7f29dc00b410 comp rx=0 tx=0).stop 2024-05-03T17:55:53.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 -- 172.21.15.122:0/2685489666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29ec107760 msgr2=0x7f29ec1a59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:53.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29ec107760 0x7f29ec1a59e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f29e4007ee0 tx=0x7f29e400d3b0 comp rx=0 tx=0).stop 2024-05-03T17:55:53.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 -- 172.21.15.122:0/2685489666 shutdown_connections 2024-05-03T17:55:53.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f29d80b2990 0x7f29d80b4e50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:53.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29ec107760 0x7f29ec1a59e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:53.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29ec102e00 0x7f29ec19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:53.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 --2- 172.21.15.122:0/2685489666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ec1037b0 0x7f29ec19aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:53.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 -- 172.21.15.122:0/2685489666 >> 172.21.15.122:0/2685489666 conn(0x7f29ec078a10 msgr2=0x7f29ec0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:53.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 -- 172.21.15.122:0/2685489666 shutdown_connections 2024-05-03T17:55:53.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:53.574+0000 7f29f1315700 1 -- 172.21.15.122:0/2685489666 wait complete. 2024-05-03T17:55:54.216 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-03T17:55:54.217 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-03T17:55:54.217 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-05-03T17:55:54.217 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-03T17:55:54.217 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-03T17:55:54.217 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-03T17:55:54.217 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-03T17:55:54.217 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-03T17:55:54.217 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-03T17:55:54.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T17:55:54.217 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-05-03T17:55:54.233 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-03T17:55:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-03T17:55:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-05-03T17:55:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-05-03T17:55:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-05-03T17:55:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-05-03T17:55:54.233 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T17:55:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42005sec preferred_lft 42005sec 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::1da6:cc88:1934:4e50/64 scope link noprefixroute 2024-05-03T17:55:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-05-03T17:55:54.235 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-03T17:55:54.235 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T17:55:54.235 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-05-03T17:55:54.235 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add name ceph-brx type bridge 2024-05-03T17:55:54.235 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr flush dev ceph-brx 2024-05-03T17:55:54.235 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set ceph-brx up 2024-05-03T17:55:54.235 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-03T17:55:54.235 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-05-03T17:55:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T17:55:54.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T17:55:54.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T17:55:54.433 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-03T17:55:54.456 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-05-03T17:55:54.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T17:55:54.458 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-05-03T17:55:54.531 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-05-03T17:55:54.531 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-03T17:55:54.531 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-03T17:55:54.531 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-03T17:55:54.532 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-03T17:55:54.532 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T17:55:54.532 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-05-03T17:55:54.532 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-03T17:55:54.532 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-03T17:55:54.532 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-03T17:55:54.532 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-05-03T17:55:54.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T17:55:54.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:54 smithi122 ceph-mon[29744]: pgmap v386: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:54.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:54 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2685489666' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T17:55:54.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:54 smithi149 ceph-mon[33409]: pgmap v386: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:54.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:54 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2685489666' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T17:55:54.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T17:55:54.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T17:55:54.829 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-05-03T17:55:54.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T17:55:54.844 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list-id 2024-05-03T17:55:54.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:54 smithi192 ceph-mon[33753]: pgmap v386: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:55:54.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:54 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2685489666' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T17:55:54.899 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T17:55:54.899 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-05-03T17:55:54.899 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-03T17:55:54.899 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-03T17:55:54.899 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-05-03T17:55:54.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T17:55:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T17:55:55.014 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-03T17:55:55.014 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T17:55:55.014 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-05-03T17:55:55.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-03T17:55:55.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-03T17:55:55.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-03T17:55:55.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-03T17:55:55.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-03T17:55:55.015 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-05-03T17:55:55.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T17:55:55.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T17:55:55.138 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T17:55:55.138 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-05-03T17:55:55.138 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-05-03T17:55:55.138 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set dev brx.0 master ceph-brx 2024-05-03T17:55:55.138 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-05-03T17:55:55.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T17:55:55.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T17:55:55.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T17:55:55.212 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-03T17:55:55.226 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-03T17:55:55.226 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T17:55:55.227 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-03T17:55:55.281 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-03T17:55:55.281 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-03T17:55:55.281 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/c64844e0-c7be-4cc0-80e5-d69d525a340f 2024-05-03T17:55:55.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T17:55:55.281 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/c64844e0-c7be-4cc0-80e5-d69d525a340f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-05-03T17:55:55.375 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-05-03T17:55:55.375 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-05-03T17:55:55.375 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-05-03T17:55:55.506 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-05-03T17:55:55.506 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-05-03T17:55:55.506 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-05-03T17:55:55.506 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.149:3300,172.21.15.192:3300" 2024-05-03T17:55:55.506 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:3300,172.21.15.149:3300,172.21.15.192:3300:/volumes/qa/sv_1/c64844e0-c7be-4cc0-80e5-d69d525a340f 2024-05-03T17:55:55.507 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=a46fc846-0974-11ef-bc97-c7b262605968" will pass to kernel 2024-05-03T17:55:55.507 INFO:teuthology.orchestra.run.smithi122.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-03T17:55:55.508 INFO:teuthology.orchestra.run.smithi122.stdout: You just mounted a file system that supports labels which does not 2024-05-03T17:55:55.508 INFO:teuthology.orchestra.run.smithi122.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-03T17:55:55.508 INFO:teuthology.orchestra.run.smithi122.stdout: applications will generate AVC messages and not be allowed access to 2024-05-03T17:55:55.508 INFO:teuthology.orchestra.run.smithi122.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-03T17:55:55.509 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-03T17:55:55.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T17:55:55.510 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-03T17:55:55.538 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T17:55:55.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T17:55:55.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T17:55:55.633 INFO:teuthology.orchestra.run.smithi122.stdout:client15177 2024-05-03T17:55:55.634 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T17:55:55.634 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T17:55:55.650 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T17:55:55.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T17:55:55.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T17:55:55.748 INFO:teuthology.orchestra.run.smithi122.stdout:client15177 2024-05-03T17:55:55.748 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T17:55:55.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T17:55:55.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T17:55:55.804 INFO:teuthology.orchestra.run.smithi122.stdout:client15177 2024-05-03T17:55:55.804 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a46fc846-0974-11ef-bc97-c7b262605968.client15177/status') 2024-05-03T17:55:55.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T17:55:55.861 INFO:teuthology.orchestra.run.smithi122.stdout:instance: client.15177 (3)192.168.0.1:0/1097069539 2024-05-03T17:55:55.861 INFO:teuthology.orchestra.run.smithi122.stdout:blocklisted: false 2024-05-03T17:55:55.861 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2024-05-03T17:55:55.862 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2024-05-03T17:55:55.862 INFO:teuthology.orchestra.run.smithi122.stderr:70 bytes copied, 2.509e-05 s, 2.8 MB/s 2024-05-03T17:55:55.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T17:55:55.869 INFO:teuthology.run_tasks:Running task exec... 2024-05-03T17:55:55.880 INFO:teuthology.task.exec:Executing custom commands... 2024-05-03T17:55:55.880 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-05-03T17:55:55.880 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-03T17:55:55.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.985+0000 7f53888a2640 1 -- 172.21.15.122:0/105103450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5384102b00 msgr2=0x7f5384102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:55.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.985+0000 7f53888a2640 1 --2- 172.21.15.122:0/105103450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5384102b00 0x7f5384102f80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5374009900 tx=0x7f537402a410 comp rx=0 tx=0).stop 2024-05-03T17:55:55.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.985+0000 7f53888a2640 1 -- 172.21.15.122:0/105103450 shutdown_connections 2024-05-03T17:55:55.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.985+0000 7f53888a2640 1 --2- 172.21.15.122:0/105103450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53841034c0 0x7f5384109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:55.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.985+0000 7f53888a2640 1 --2- 172.21.15.122:0/105103450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5384102b00 0x7f5384102f80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:55.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.985+0000 7f53888a2640 1 --2- 172.21.15.122:0/105103450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5384101910 0x7f5384101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:55.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.985+0000 7f53888a2640 1 -- 172.21.15.122:0/105103450 >> 172.21.15.122:0/105103450 conn(0x7f53840fcea0 msgr2=0x7f53840ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:55.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.985+0000 7f53888a2640 1 -- 172.21.15.122:0/105103450 shutdown_connections 2024-05-03T17:55:55.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.985+0000 7f53888a2640 1 -- 172.21.15.122:0/105103450 wait complete. 2024-05-03T17:55:55.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f53888a2640 1 Processor -- start 2024-05-03T17:55:55.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f53888a2640 1 -- start start 2024-05-03T17:55:55.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f53888a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5384101910 0x7f53841058a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:55.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f53888a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5384102b00 0x7f5384103ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:55.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f53888a2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53841034c0 0x7f5384104420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:55.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f53888a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f538410d420 con 0x7f5384102b00 2024-05-03T17:55:55.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f53888a2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f538410d2a0 con 0x7f53841034c0 2024-05-03T17:55:55.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f53888a2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f538410d5a0 con 0x7f5384101910 2024-05-03T17:55:55.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f5381d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5384102b00 0x7f5384103ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:55.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f5381d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5384102b00 0x7f5384103ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45238/0 (socket says 172.21.15.122:45238) 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f5381d9b640 1 -- 172.21.15.122:0/356956497 learned_addr learned my addr 172.21.15.122:0/356956497 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f5382d9d640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53841034c0 0x7f5384104420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.986+0000 7f538259c640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5384101910 0x7f53841058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.987+0000 7f5381d9b640 1 -- 172.21.15.122:0/356956497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5384101910 msgr2=0x7f53841058a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.987+0000 7f5381d9b640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5384101910 0x7f53841058a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.987+0000 7f5381d9b640 1 -- 172.21.15.122:0/356956497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53841034c0 msgr2=0x7f5384104420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.987+0000 7f5381d9b640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53841034c0 0x7f5384104420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.987+0000 7f5381d9b640 1 -- 172.21.15.122:0/356956497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5384104cb0 con 0x7f5384102b00 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.987+0000 7f5381d9b640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5384102b00 0x7f5384103ee0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5374005940 tx=0x7f5374004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.987+0000 7f53737fe640 1 -- 172.21.15.122:0/356956497 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537403f030 con 0x7f5384102b00 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.987+0000 7f53737fe640 1 -- 172.21.15.122:0/356956497 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5374003cb0 con 0x7f5384102b00 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.987+0000 7f53888a2640 1 -- 172.21.15.122:0/356956497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53841a2b90 con 0x7f5384102b00 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.987+0000 7f53737fe640 1 -- 172.21.15.122:0/356956497 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374035070 con 0x7f5384102b00 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.987+0000 7f53888a2640 1 -- 172.21.15.122:0/356956497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53841a30e0 con 0x7f5384102b00 2024-05-03T17:55:55.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.988+0000 7f53888a2640 1 -- 172.21.15.122:0/356956497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53841a32b0 con 0x7f5384102b00 2024-05-03T17:55:55.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.990+0000 7f53737fe640 1 -- 172.21.15.122:0/356956497 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f5374042030 con 0x7f5384102b00 2024-05-03T17:55:55.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.991+0000 7f53737fe640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f536c0b2920 0x7f536c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:55.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.991+0000 7f538259c640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f536c0b2920 0x7f536c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:55.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.991+0000 7f53737fe640 1 -- 172.21.15.122:0/356956497 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f5374102d90 con 0x7f5384102b00 2024-05-03T17:55:55.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.991+0000 7f538259c640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f536c0b2920 0x7f536c0b4de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5384102960 tx=0x7f5378005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:55.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:55.994+0000 7f53737fe640 1 -- 172.21.15.122:0/356956497 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5374003e60 con 0x7f5384102b00 2024-05-03T17:55:56.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.220+0000 7f53888a2640 1 -- 172.21.15.122:0/356956497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f53841a3480 con 0x7f5384102b00 2024-05-03T17:55:56.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.517+0000 7f53737fe640 1 -- 172.21.15.122:0/356956497 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f53740ca400 con 0x7f5384102b00 2024-05-03T17:55:56.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 -- 172.21.15.122:0/356956497 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f536c0b2920 msgr2=0x7f536c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:56.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f536c0b2920 0x7f536c0b4de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5384102960 tx=0x7f5378005f90 comp rx=0 tx=0).stop 2024-05-03T17:55:56.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 -- 172.21.15.122:0/356956497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5384102b00 msgr2=0x7f5384103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:56.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5384102b00 0x7f5384103ee0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5374005940 tx=0x7f5374004430 comp rx=0 tx=0).stop 2024-05-03T17:55:56.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 -- 172.21.15.122:0/356956497 shutdown_connections 2024-05-03T17:55:56.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f536c0b2920 0x7f536c0b4de0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:56.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53841034c0 0x7f5384104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:56.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5384102b00 0x7f5384103ee0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:56.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 --2- 172.21.15.122:0/356956497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5384101910 0x7f53841058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:56.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 -- 172.21.15.122:0/356956497 >> 172.21.15.122:0/356956497 conn(0x7f53840fcea0 msgr2=0x7f53840fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:56.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 -- 172.21.15.122:0/356956497 shutdown_connections 2024-05-03T17:55:56.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.520+0000 7f53888a2640 1 -- 172.21.15.122:0/356956497 wait complete. 2024-05-03T17:55:56.527 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-03T17:55:56.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.635+0000 7fbf44b04640 1 -- 172.21.15.122:0/2093556542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf401034c0 msgr2=0x7fbf40109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:56.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.635+0000 7fbf44b04640 1 --2- 172.21.15.122:0/2093556542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf401034c0 0x7fbf40109d50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009ca0 tx=0x7fbf3802a8c0 comp rx=0 tx=0).stop 2024-05-03T17:55:56.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.635+0000 7fbf44b04640 1 -- 172.21.15.122:0/2093556542 shutdown_connections 2024-05-03T17:55:56.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.635+0000 7fbf44b04640 1 --2- 172.21.15.122:0/2093556542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf401034c0 0x7fbf40109d50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:56.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.635+0000 7fbf44b04640 1 --2- 172.21.15.122:0/2093556542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf40102b00 0x7fbf40102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:56.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.635+0000 7fbf44b04640 1 --2- 172.21.15.122:0/2093556542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40101910 0x7fbf40101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:56.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.635+0000 7fbf44b04640 1 -- 172.21.15.122:0/2093556542 >> 172.21.15.122:0/2093556542 conn(0x7fbf400fce80 msgr2=0x7fbf400ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T17:55:56.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.635+0000 7fbf44b04640 1 -- 172.21.15.122:0/2093556542 shutdown_connections 2024-05-03T17:55:56.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.635+0000 7fbf44b04640 1 -- 172.21.15.122:0/2093556542 wait complete. 2024-05-03T17:55:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf44b04640 1 Processor -- start 2024-05-03T17:55:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf44b04640 1 -- start start 2024-05-03T17:55:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf44b04640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf40101910 0x7fbf40197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf44b04640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40102b00 0x7fbf40197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf44b04640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf401034c0 0x7fbf4019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf44b04640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4010d4b0 con 0x7fbf401034c0 2024-05-03T17:55:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf44b04640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4010d330 con 0x7fbf40102b00 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf44b04640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4010d630 con 0x7fbf40101910 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf3ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf401034c0 0x7fbf4019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf3ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf401034c0 0x7fbf4019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45264/0 (socket says 172.21.15.122:45264) 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf3ed9d640 1 -- 172.21.15.122:0/2495789089 learned_addr learned my addr 172.21.15.122:0/2495789089 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf3e59c640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf40101910 0x7fbf40197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf37fff640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40102b00 0x7fbf40197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf3ed9d640 1 -- 172.21.15.122:0/2495789089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf40101910 msgr2=0x7fbf40197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf3ed9d640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf40101910 0x7fbf40197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf3ed9d640 1 -- 172.21.15.122:0/2495789089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40102b00 msgr2=0x7fbf40197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf3ed9d640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40102b00 0x7fbf40197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf3ed9d640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4019f650 con 0x7fbf401034c0 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf37fff640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40102b00 0x7fbf40197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T17:55:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf3e59c640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf40101910 0x7fbf40197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T17:55:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.636+0000 7fbf3ed9d640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf401034c0 0x7fbf4019efe0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbf38031a20 tx=0x7fbf38003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.637+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38004240 con 0x7fbf401034c0 2024-05-03T17:55:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.637+0000 7fbf44b04640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4019f8e0 con 0x7fbf401034c0 2024-05-03T17:55:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.637+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf38003e10 con 0x7fbf401034c0 2024-05-03T17:55:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.637+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf380305c0 con 0x7fbf401034c0 2024-05-03T17:55:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.637+0000 7fbf44b04640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4019ff00 con 0x7fbf401034c0 2024-05-03T17:55:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.638+0000 7fbf44b04640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf40108b60 con 0x7fbf401034c0 2024-05-03T17:55:56.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.639+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fbf3802f660 con 0x7fbf401034c0 2024-05-03T17:55:56.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.639+0000 7fbf377fe640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf2c0b2920 0x7fbf2c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T17:55:56.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.640+0000 7fbf3e59c640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf2c0b2920 0x7fbf2c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T17:55:56.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.640+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fbf38103c80 con 0x7fbf401034c0 2024-05-03T17:55:56.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.640+0000 7fbf3e59c640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf2c0b2920 0x7fbf2c0b4de0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbf40102960 tx=0x7fbf30008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T17:55:56.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.643+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbf380cba50 con 0x7fbf401034c0 2024-05-03T17:55:56.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:56 smithi149 ceph-mon[33409]: pgmap v387: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:56.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:56 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/356956497' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T17:55:56.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:55:56.855+0000 7fbf44b04640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fbf40062e90 con 0x7fbf2c0b2920 2024-05-03T17:55:56.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:56 smithi192 ceph-mon[33753]: pgmap v387: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:56.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:56 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/356956497' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T17:55:56.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:56 smithi122 ceph-mon[29744]: pgmap v387: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:55:56.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:56 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/356956497' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T17:55:57.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:57 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/356956497' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T17:55:57.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:57 smithi149 ceph-mon[33409]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T17:55:57.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:57 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/356956497' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T17:55:57.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:57 smithi192 ceph-mon[33753]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T17:55:57.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:57 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/356956497' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T17:55:57.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:57 smithi122 ceph-mon[29744]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T17:55:58.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:58 smithi149 ceph-mon[33409]: pgmap v388: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:58.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:55:58 smithi149 ceph-mon[33409]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:58.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:58 smithi192 ceph-mon[33753]: pgmap v388: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:58.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:55:58 smithi192 ceph-mon[33753]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:55:58.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:58 smithi122 ceph-mon[29744]: pgmap v388: 97 pgs: 97 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:55:58.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:55:58 smithi122 ceph-mon[29744]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T17:56:00.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:00 smithi149 ceph-mon[33409]: pgmap v389: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-03T17:56:00.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:00 smithi192 ceph-mon[33753]: pgmap v389: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-03T17:56:00.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:00 smithi122 ceph-mon[29744]: pgmap v389: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-03T17:56:02.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:02 smithi149 ceph-mon[33409]: pgmap v390: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-03T17:56:02.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:02 smithi192 ceph-mon[33753]: pgmap v390: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-03T17:56:02.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:02 smithi122 ceph-mon[29744]: pgmap v390: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-03T17:56:04.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:04 smithi192 ceph-mon[33753]: pgmap v391: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-03T17:56:04.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:04 smithi122 ceph-mon[29744]: pgmap v391: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-03T17:56:05.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:04 smithi149 ceph-mon[33409]: pgmap v391: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-03T17:56:06.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:06 smithi192 ceph-mon[33753]: pgmap v392: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-03T17:56:06.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:06 smithi122 ceph-mon[29744]: pgmap v392: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-03T17:56:07.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:06 smithi149 ceph-mon[33409]: pgmap v392: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-03T17:56:08.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:08 smithi192 ceph-mon[33753]: pgmap v393: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-03T17:56:08.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:08 smithi122 ceph-mon[29744]: pgmap v393: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-03T17:56:09.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:08 smithi149 ceph-mon[33409]: pgmap v393: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-03T17:56:10.839 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:10 smithi149 ceph-mon[33409]: pgmap v394: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-03T17:56:10.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:10 smithi192 ceph-mon[33753]: pgmap v394: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-03T17:56:10.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:10 smithi122 ceph-mon[29744]: pgmap v394: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-03T17:56:11.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:11 smithi192 ceph-mon[33753]: pgmap v395: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T17:56:11.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:11 smithi122 ceph-mon[29744]: pgmap v395: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T17:56:12.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:11 smithi149 ceph-mon[33409]: pgmap v395: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T17:56:13.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:13 smithi192 ceph-mon[33753]: pgmap v396: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:13.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:13 smithi122 ceph-mon[29744]: pgmap v396: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:14.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:13 smithi149 ceph-mon[33409]: pgmap v396: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:15.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:15 smithi192 ceph-mon[33753]: pgmap v397: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:15.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:15 smithi122 ceph-mon[29744]: pgmap v397: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:16.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:15 smithi149 ceph-mon[33409]: pgmap v397: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:17.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:17 smithi192 ceph-mon[33753]: pgmap v398: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:17.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:17 smithi122 ceph-mon[29744]: pgmap v398: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:18.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:17 smithi149 ceph-mon[33409]: pgmap v398: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:20.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:19 smithi192 ceph-mon[33753]: pgmap v399: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:20.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:19 smithi122 ceph-mon[29744]: pgmap v399: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:20.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:19 smithi149 ceph-mon[33409]: pgmap v399: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:22.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:21 smithi192 ceph-mon[33753]: pgmap v400: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:22.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:21 smithi122 ceph-mon[29744]: pgmap v400: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:22.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:21 smithi149 ceph-mon[33409]: pgmap v400: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:24.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:23 smithi192 ceph-mon[33753]: pgmap v401: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:24.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:23 smithi122 ceph-mon[29744]: pgmap v401: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:24.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:23 smithi149 ceph-mon[33409]: pgmap v401: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:26.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:25 smithi192 ceph-mon[33753]: pgmap v402: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:26.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:25 smithi122 ceph-mon[29744]: pgmap v402: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:26.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:25 smithi149 ceph-mon[33409]: pgmap v402: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:28.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:27 smithi192 ceph-mon[33753]: pgmap v403: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:28.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:27 smithi122 ceph-mon[29744]: pgmap v403: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:28.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:27 smithi149 ceph-mon[33409]: pgmap v403: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:30.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:29 smithi192 ceph-mon[33753]: pgmap v404: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:30.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:29 smithi122 ceph-mon[29744]: pgmap v404: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:30.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:29 smithi149 ceph-mon[33409]: pgmap v404: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:32.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:31 smithi192 ceph-mon[33753]: pgmap v405: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:32.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:31 smithi122 ceph-mon[29744]: pgmap v405: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:32.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:31 smithi149 ceph-mon[33409]: pgmap v405: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:34.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:33 smithi192 ceph-mon[33753]: pgmap v406: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:34.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:33 smithi122 ceph-mon[29744]: pgmap v406: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:34.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:33 smithi149 ceph-mon[33409]: pgmap v406: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:36.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:35 smithi192 ceph-mon[33753]: pgmap v407: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:36.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:35 smithi122 ceph-mon[29744]: pgmap v407: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:36.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:35 smithi149 ceph-mon[33409]: pgmap v407: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:38.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:37 smithi192 ceph-mon[33753]: pgmap v408: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:38.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:37 smithi122 ceph-mon[29744]: pgmap v408: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:38.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:37 smithi149 ceph-mon[33409]: pgmap v408: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:40.088 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:39 smithi149 ceph-mon[33409]: pgmap v409: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:40.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:39 smithi192 ceph-mon[33753]: pgmap v409: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:40.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:39 smithi122 ceph-mon[29744]: pgmap v409: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:42.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:41 smithi192 ceph-mon[33753]: pgmap v410: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:42.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:41 smithi122 ceph-mon[29744]: pgmap v410: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:42.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:41 smithi149 ceph-mon[33409]: pgmap v410: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:44.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:43 smithi192 ceph-mon[33753]: pgmap v411: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:44.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:43 smithi122 ceph-mon[29744]: pgmap v411: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:44.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:43 smithi149 ceph-mon[33409]: pgmap v411: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:46.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:45 smithi192 ceph-mon[33753]: pgmap v412: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:46.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:45 smithi122 ceph-mon[29744]: pgmap v412: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:46.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:45 smithi149 ceph-mon[33409]: pgmap v412: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:48.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:47 smithi192 ceph-mon[33753]: pgmap v413: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:48.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:47 smithi122 ceph-mon[29744]: pgmap v413: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:48.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:47 smithi149 ceph-mon[33409]: pgmap v413: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:50.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:49 smithi192 ceph-mon[33753]: pgmap v414: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:50.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:49 smithi122 ceph-mon[29744]: pgmap v414: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:50.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:49 smithi149 ceph-mon[33409]: pgmap v414: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:56:51.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:50 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:56:51.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:50 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:56:51.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:50 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:56:52.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:52 smithi192 ceph-mon[33753]: pgmap v415: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:52.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:56:52.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:56:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:56:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:56:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:56:52.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:52 smithi122 ceph-mon[29744]: pgmap v415: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:52.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:56:52.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:56:52.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:56:52.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:56:52.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:56:52.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:52 smithi149 ceph-mon[33409]: pgmap v415: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:56:52.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:56:52.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:56:52.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:56:52.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:56:52.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:56:54.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:54 smithi192 ceph-mon[33753]: pgmap v416: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:54.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:54 smithi122 ceph-mon[29744]: pgmap v416: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:54.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:54 smithi149 ceph-mon[33409]: pgmap v416: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:56:56.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:56 smithi192 ceph-mon[33753]: pgmap v417: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:56.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:56 smithi122 ceph-mon[29744]: pgmap v417: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:56.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:56 smithi149 ceph-mon[33409]: pgmap v417: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:56:58.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:56:58 smithi192 ceph-mon[33753]: pgmap v418: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:58.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:56:58 smithi122 ceph-mon[29744]: pgmap v418: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:56:58.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:56:58 smithi149 ceph-mon[33409]: pgmap v418: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:00.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:00 smithi192 ceph-mon[33753]: pgmap v419: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:00.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:00 smithi122 ceph-mon[29744]: pgmap v419: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:00.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:00 smithi149 ceph-mon[33409]: pgmap v419: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:02.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:02 smithi122 ceph-mon[29744]: pgmap v420: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:02.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:02 smithi149 ceph-mon[33409]: pgmap v420: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:02.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:02 smithi192 ceph-mon[33753]: pgmap v420: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:04.667 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:04 smithi192 ceph-mon[33753]: pgmap v421: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:04.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:04 smithi122 ceph-mon[29744]: pgmap v421: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:04.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:04 smithi149 ceph-mon[33409]: pgmap v421: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:06.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:06 smithi122 ceph-mon[29744]: pgmap v422: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:06.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:06 smithi149 ceph-mon[33409]: pgmap v422: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:06.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:06 smithi192 ceph-mon[33753]: pgmap v422: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:08.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:08 smithi122 ceph-mon[29744]: pgmap v423: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:57:08.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:08 smithi149 ceph-mon[33409]: pgmap v423: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:57:08.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:08 smithi192 ceph-mon[33753]: pgmap v423: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:57:10.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:10 smithi122 ceph-mon[29744]: pgmap v424: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:10.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:10 smithi149 ceph-mon[33409]: pgmap v424: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:10.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:10 smithi192 ceph-mon[33753]: pgmap v424: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:12.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:12 smithi122 ceph-mon[29744]: pgmap v425: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:57:12.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:12 smithi149 ceph-mon[33409]: pgmap v425: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:57:12.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:12 smithi192 ceph-mon[33753]: pgmap v425: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:57:14.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:14 smithi122 ceph-mon[29744]: pgmap v426: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:14.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:14 smithi149 ceph-mon[33409]: pgmap v426: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:14.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:14 smithi192 ceph-mon[33753]: pgmap v426: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:57:16.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:16 smithi122 ceph-mon[29744]: pgmap v427: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:16.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:16 smithi149 ceph-mon[33409]: pgmap v427: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:16.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:16 smithi192 ceph-mon[33753]: pgmap v427: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:18.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:18 smithi122 ceph-mon[29744]: pgmap v428: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:18.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:18 smithi149 ceph-mon[33409]: pgmap v428: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:18.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:18 smithi192 ceph-mon[33753]: pgmap v428: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:20.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:20 smithi122 ceph-mon[29744]: pgmap v429: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:20.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:20 smithi149 ceph-mon[33409]: pgmap v429: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:20.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:20 smithi192 ceph-mon[33753]: pgmap v429: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:22.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:22 smithi122 ceph-mon[29744]: pgmap v430: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:22.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:22 smithi149 ceph-mon[33409]: pgmap v430: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:22.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:22 smithi192 ceph-mon[33753]: pgmap v430: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:24.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:24 smithi122 ceph-mon[29744]: pgmap v431: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:24.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:24 smithi149 ceph-mon[33409]: pgmap v431: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:24.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:24 smithi192 ceph-mon[33753]: pgmap v431: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:26.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:26 smithi122 ceph-mon[29744]: pgmap v432: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:26.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:26 smithi149 ceph-mon[33409]: pgmap v432: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:26.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:26 smithi192 ceph-mon[33753]: pgmap v432: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:28.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:28 smithi122 ceph-mon[29744]: pgmap v433: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:28.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:28 smithi149 ceph-mon[33409]: pgmap v433: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:28.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:28 smithi192 ceph-mon[33753]: pgmap v433: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:30.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:30 smithi122 ceph-mon[29744]: pgmap v434: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:30.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:30 smithi149 ceph-mon[33409]: pgmap v434: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:30.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:30 smithi192 ceph-mon[33753]: pgmap v434: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:32.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:32 smithi122 ceph-mon[29744]: pgmap v435: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:32 smithi149 ceph-mon[33409]: pgmap v435: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:32.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:32 smithi192 ceph-mon[33753]: pgmap v435: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:34.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:34 smithi122 ceph-mon[29744]: pgmap v436: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:34.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:34 smithi149 ceph-mon[33409]: pgmap v436: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:34.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:34 smithi192 ceph-mon[33753]: pgmap v436: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:36.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:36 smithi122 ceph-mon[29744]: pgmap v437: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:36.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:36 smithi149 ceph-mon[33409]: pgmap v437: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:36.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:36 smithi192 ceph-mon[33753]: pgmap v437: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:38.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:38 smithi122 ceph-mon[29744]: pgmap v438: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:38.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:38 smithi149 ceph-mon[33409]: pgmap v438: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:38.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:38 smithi192 ceph-mon[33753]: pgmap v438: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:40.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:40 smithi122 ceph-mon[29744]: pgmap v439: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:40.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:40 smithi149 ceph-mon[33409]: pgmap v439: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:40.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:40 smithi192 ceph-mon[33753]: pgmap v439: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:42.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:42 smithi122 ceph-mon[29744]: pgmap v440: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:42.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:42 smithi149 ceph-mon[33409]: pgmap v440: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:42.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:42 smithi192 ceph-mon[33753]: pgmap v440: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:44.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:44 smithi122 ceph-mon[29744]: pgmap v441: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:44.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:44 smithi149 ceph-mon[33409]: pgmap v441: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:44.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:44 smithi192 ceph-mon[33753]: pgmap v441: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:46.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:46 smithi122 ceph-mon[29744]: pgmap v442: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:46.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:46 smithi149 ceph-mon[33409]: pgmap v442: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:46.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:46 smithi192 ceph-mon[33753]: pgmap v442: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:48.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:48 smithi122 ceph-mon[29744]: pgmap v443: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:48.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:48 smithi149 ceph-mon[33409]: pgmap v443: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:48.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:48 smithi192 ceph-mon[33753]: pgmap v443: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:50.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:50 smithi122 ceph-mon[29744]: pgmap v444: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:50.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:50 smithi149 ceph-mon[33409]: pgmap v444: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:50.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:50 smithi192 ceph-mon[33753]: pgmap v444: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:57:52.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:52 smithi122 ceph-mon[29744]: pgmap v445: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:52.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:52 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:57:52.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:52 smithi149 ceph-mon[33409]: pgmap v445: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:52.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:52 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:57:52.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:52 smithi192 ceph-mon[33753]: pgmap v445: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:57:52.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:52 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:57:53.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:53 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:57:53.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:53 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:57:53.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:53 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:57:53.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:53 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:57:53.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:53 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:57:53.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:53 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:57:53.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:53 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:57:53.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:53 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:57:53.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:53 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:57:54.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:54 smithi122 ceph-mon[29744]: pgmap v446: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:54.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:54 smithi149 ceph-mon[33409]: pgmap v446: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:54.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:54 smithi192 ceph-mon[33753]: pgmap v446: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:57:56.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:56 smithi149 ceph-mon[33409]: pgmap v447: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:56.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:56 smithi192 ceph-mon[33753]: pgmap v447: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:56.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:56 smithi122 ceph-mon[29744]: pgmap v447: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:57:58.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:57:58 smithi149 ceph-mon[33409]: pgmap v448: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:58.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:57:58 smithi192 ceph-mon[33753]: pgmap v448: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:57:58.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:57:58 smithi122 ceph-mon[29744]: pgmap v448: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:00.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:00 smithi149 ceph-mon[33409]: pgmap v449: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:00.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:00 smithi192 ceph-mon[33753]: pgmap v449: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:00.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:00 smithi122 ceph-mon[29744]: pgmap v449: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:02.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:02 smithi149 ceph-mon[33409]: pgmap v450: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:58:02.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:02 smithi192 ceph-mon[33753]: pgmap v450: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:58:02.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:02 smithi122 ceph-mon[29744]: pgmap v450: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:58:04.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:04 smithi149 ceph-mon[33409]: pgmap v451: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:58:04.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:04 smithi192 ceph-mon[33753]: pgmap v451: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:58:04.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:04 smithi122 ceph-mon[29744]: pgmap v451: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:58:06.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:06 smithi149 ceph-mon[33409]: pgmap v452: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:58:06.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:06 smithi192 ceph-mon[33753]: pgmap v452: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:58:06.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:06 smithi122 ceph-mon[29744]: pgmap v452: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:58:08.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:08 smithi149 ceph-mon[33409]: pgmap v453: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:58:08.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:08 smithi192 ceph-mon[33753]: pgmap v453: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:58:08.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:08 smithi122 ceph-mon[29744]: pgmap v453: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:58:10.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:10 smithi149 ceph-mon[33409]: pgmap v454: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:58:10.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:10 smithi192 ceph-mon[33753]: pgmap v454: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:58:10.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:10 smithi122 ceph-mon[29744]: pgmap v454: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-03T17:58:12.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:12 smithi149 ceph-mon[33409]: pgmap v455: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:58:12.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:12 smithi192 ceph-mon[33753]: pgmap v455: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:58:12.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:12 smithi122 ceph-mon[29744]: pgmap v455: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-03T17:58:14.762 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:14 smithi122 ceph-mon[29744]: pgmap v456: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:14.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:14 smithi149 ceph-mon[33409]: pgmap v456: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:14.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:14 smithi192 ceph-mon[33753]: pgmap v456: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:16.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:16 smithi149 ceph-mon[33409]: pgmap v457: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:16.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:16 smithi192 ceph-mon[33753]: pgmap v457: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:16.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:16 smithi122 ceph-mon[29744]: pgmap v457: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:18.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:18 smithi149 ceph-mon[33409]: pgmap v458: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:18.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:18 smithi192 ceph-mon[33753]: pgmap v458: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:18.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:18 smithi122 ceph-mon[29744]: pgmap v458: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:20.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:20 smithi149 ceph-mon[33409]: pgmap v459: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:20.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:20 smithi192 ceph-mon[33753]: pgmap v459: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:20.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:20 smithi122 ceph-mon[29744]: pgmap v459: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:22.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:22 smithi149 ceph-mon[33409]: pgmap v460: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:22.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:22 smithi192 ceph-mon[33753]: pgmap v460: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:22.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:22 smithi122 ceph-mon[29744]: pgmap v460: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:24.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:24 smithi149 ceph-mon[33409]: pgmap v461: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:24.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:24 smithi192 ceph-mon[33753]: pgmap v461: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:24.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:24 smithi122 ceph-mon[29744]: pgmap v461: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:26.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:26 smithi149 ceph-mon[33409]: pgmap v462: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:26.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:26 smithi192 ceph-mon[33753]: pgmap v462: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:26.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:26 smithi122 ceph-mon[29744]: pgmap v462: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:28.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:28 smithi149 ceph-mon[33409]: pgmap v463: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:28.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:28 smithi192 ceph-mon[33753]: pgmap v463: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:28.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:28 smithi122 ceph-mon[29744]: pgmap v463: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:30.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:30 smithi149 ceph-mon[33409]: pgmap v464: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:30.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:30 smithi192 ceph-mon[33753]: pgmap v464: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:30.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:30 smithi122 ceph-mon[29744]: pgmap v464: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:32 smithi149 ceph-mon[33409]: pgmap v465: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:32.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:32 smithi192 ceph-mon[33753]: pgmap v465: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:32.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:32 smithi122 ceph-mon[29744]: pgmap v465: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:34.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:34 smithi149 ceph-mon[33409]: pgmap v466: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:34.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:34 smithi192 ceph-mon[33753]: pgmap v466: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:34.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:34 smithi122 ceph-mon[29744]: pgmap v466: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:36.787 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:36 smithi149 ceph-mon[33409]: pgmap v467: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:36.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:36 smithi192 ceph-mon[33753]: pgmap v467: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:36.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:36 smithi122 ceph-mon[29744]: pgmap v467: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:38.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:38 smithi149 ceph-mon[33409]: pgmap v468: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:38.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:38 smithi192 ceph-mon[33753]: pgmap v468: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:38.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:38 smithi122 ceph-mon[29744]: pgmap v468: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:40.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:40 smithi149 ceph-mon[33409]: pgmap v469: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:40.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:40 smithi192 ceph-mon[33753]: pgmap v469: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:40.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:40 smithi122 ceph-mon[29744]: pgmap v469: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:42.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:42 smithi149 ceph-mon[33409]: pgmap v470: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:42.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:42 smithi192 ceph-mon[33753]: pgmap v470: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:42.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:42 smithi122 ceph-mon[29744]: pgmap v470: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:44.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:44 smithi192 ceph-mon[33753]: pgmap v471: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:44.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:44 smithi122 ceph-mon[29744]: pgmap v471: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:45.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:44 smithi149 ceph-mon[33409]: pgmap v471: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:46.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:46 smithi192 ceph-mon[33753]: pgmap v472: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:46.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:46 smithi122 ceph-mon[29744]: pgmap v472: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:47.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:46 smithi149 ceph-mon[33409]: pgmap v472: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:48.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:48 smithi192 ceph-mon[33753]: pgmap v473: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:48.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:48 smithi122 ceph-mon[29744]: pgmap v473: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:49.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:48 smithi149 ceph-mon[33409]: pgmap v473: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:50.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:50 smithi192 ceph-mon[33753]: pgmap v474: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:50.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:50 smithi122 ceph-mon[29744]: pgmap v474: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:51.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:50 smithi149 ceph-mon[33409]: pgmap v474: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:58:52.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:52 smithi192 ceph-mon[33753]: pgmap v475: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:52.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:52 smithi122 ceph-mon[29744]: pgmap v475: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:53.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:52 smithi149 ceph-mon[33409]: pgmap v475: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:58:53.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:53 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:58:53.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:53 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:58:54.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:53 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:58:54.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:54 smithi192 ceph-mon[33753]: pgmap v476: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:54.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:58:54.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:58:54.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:58:54.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:54 smithi122 ceph-mon[29744]: pgmap v476: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:54.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:58:54.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:58:54.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:58:55.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:54 smithi149 ceph-mon[33409]: pgmap v476: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:58:55.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:58:55.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:58:55.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:58:56.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:58:56.660+0000 7fbf367fc640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fbf240034c0 con 0x7fbf401034c0 2024-05-03T17:58:56.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T17:58:56.661+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fbf240034c0 con 0x7fbf401034c0 2024-05-03T17:58:56.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:56 smithi192 ceph-mon[33753]: pgmap v477: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:56.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:56 smithi122 ceph-mon[29744]: pgmap v477: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:57.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:56 smithi149 ceph-mon[33409]: pgmap v477: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:58:58.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:58:58 smithi192 ceph-mon[33753]: pgmap v478: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:58.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:58:58 smithi122 ceph-mon[29744]: pgmap v478: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:58:59.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:58:58 smithi149 ceph-mon[33409]: pgmap v478: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:00.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:00 smithi192 ceph-mon[33753]: pgmap v479: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:00.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:00 smithi122 ceph-mon[29744]: pgmap v479: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:01.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:00 smithi149 ceph-mon[33409]: pgmap v479: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:01.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:01 smithi122 ceph-mon[29744]: pgmap v480: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:02.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:01 smithi149 ceph-mon[33409]: pgmap v480: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:02.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:01 smithi192 ceph-mon[33753]: pgmap v480: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:03.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:03 smithi122 ceph-mon[29744]: pgmap v481: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:04.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:03 smithi149 ceph-mon[33409]: pgmap v481: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:04.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:03 smithi192 ceph-mon[33753]: pgmap v481: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:05.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:05 smithi122 ceph-mon[29744]: pgmap v482: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:06.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:05 smithi149 ceph-mon[33409]: pgmap v482: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:06.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:05 smithi192 ceph-mon[33753]: pgmap v482: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:07.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:07 smithi122 ceph-mon[29744]: pgmap v483: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:08.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:07 smithi149 ceph-mon[33409]: pgmap v483: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:08.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:07 smithi192 ceph-mon[33753]: pgmap v483: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:09.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:09 smithi122 ceph-mon[29744]: pgmap v484: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:10.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:09 smithi149 ceph-mon[33409]: pgmap v484: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:10.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:09 smithi192 ceph-mon[33753]: pgmap v484: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:11.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:11 smithi122 ceph-mon[29744]: pgmap v485: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:12.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:11 smithi149 ceph-mon[33409]: pgmap v485: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:12.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:11 smithi192 ceph-mon[33753]: pgmap v485: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:13.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:13 smithi122 ceph-mon[29744]: pgmap v486: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:14.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:13 smithi149 ceph-mon[33409]: pgmap v486: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:14.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:13 smithi192 ceph-mon[33753]: pgmap v486: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:16.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:15 smithi149 ceph-mon[33409]: pgmap v487: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:16.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:15 smithi192 ceph-mon[33753]: pgmap v487: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:16.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:15 smithi122 ceph-mon[29744]: pgmap v487: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:18.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:17 smithi149 ceph-mon[33409]: pgmap v488: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:18.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:17 smithi192 ceph-mon[33753]: pgmap v488: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:18.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:17 smithi122 ceph-mon[29744]: pgmap v488: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:20.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:19 smithi149 ceph-mon[33409]: pgmap v489: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:20.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:19 smithi192 ceph-mon[33753]: pgmap v489: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:20.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:19 smithi122 ceph-mon[29744]: pgmap v489: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:22.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:21 smithi149 ceph-mon[33409]: pgmap v490: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:22.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:21 smithi192 ceph-mon[33753]: pgmap v490: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:22.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:21 smithi122 ceph-mon[29744]: pgmap v490: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:24.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:23 smithi149 ceph-mon[33409]: pgmap v491: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:24.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:23 smithi192 ceph-mon[33753]: pgmap v491: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:24.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:23 smithi122 ceph-mon[29744]: pgmap v491: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:26.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:25 smithi149 ceph-mon[33409]: pgmap v492: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:26.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:25 smithi192 ceph-mon[33753]: pgmap v492: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:26.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:25 smithi122 ceph-mon[29744]: pgmap v492: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:28.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:27 smithi149 ceph-mon[33409]: pgmap v493: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:28.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:27 smithi192 ceph-mon[33753]: pgmap v493: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:28.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:27 smithi122 ceph-mon[29744]: pgmap v493: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:30.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:29 smithi149 ceph-mon[33409]: pgmap v494: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:30.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:29 smithi192 ceph-mon[33753]: pgmap v494: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:30.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:29 smithi122 ceph-mon[29744]: pgmap v494: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:32.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:31 smithi149 ceph-mon[33409]: pgmap v495: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:32.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:31 smithi192 ceph-mon[33753]: pgmap v495: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:32.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:31 smithi122 ceph-mon[29744]: pgmap v495: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:34.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:33 smithi149 ceph-mon[33409]: pgmap v496: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:34.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:33 smithi192 ceph-mon[33753]: pgmap v496: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:34.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:33 smithi122 ceph-mon[29744]: pgmap v496: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:36.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:35 smithi149 ceph-mon[33409]: pgmap v497: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:36.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:35 smithi192 ceph-mon[33753]: pgmap v497: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:36.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:35 smithi122 ceph-mon[29744]: pgmap v497: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:38.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:37 smithi149 ceph-mon[33409]: pgmap v498: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:38.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:37 smithi192 ceph-mon[33753]: pgmap v498: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:38.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:37 smithi122 ceph-mon[29744]: pgmap v498: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:40.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:39 smithi149 ceph-mon[33409]: pgmap v499: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:40.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:39 smithi192 ceph-mon[33753]: pgmap v499: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:40.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:39 smithi122 ceph-mon[29744]: pgmap v499: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:42.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:41 smithi149 ceph-mon[33409]: pgmap v500: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:42.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:41 smithi192 ceph-mon[33753]: pgmap v500: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:42.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:41 smithi122 ceph-mon[29744]: pgmap v500: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:44.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:43 smithi149 ceph-mon[33409]: pgmap v501: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:44.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:43 smithi192 ceph-mon[33753]: pgmap v501: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:44.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:43 smithi122 ceph-mon[29744]: pgmap v501: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:46.010 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:45 smithi122 ceph-mon[29744]: pgmap v502: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:46.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:45 smithi149 ceph-mon[33409]: pgmap v502: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:46.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:45 smithi192 ceph-mon[33753]: pgmap v502: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:48.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:47 smithi149 ceph-mon[33409]: pgmap v503: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:48.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:47 smithi192 ceph-mon[33753]: pgmap v503: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:48.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:47 smithi122 ceph-mon[29744]: pgmap v503: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:50.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:49 smithi149 ceph-mon[33409]: pgmap v504: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:50.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:49 smithi192 ceph-mon[33753]: pgmap v504: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:50.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:49 smithi122 ceph-mon[29744]: pgmap v504: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T17:59:52.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:51 smithi149 ceph-mon[33409]: pgmap v505: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:52.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:51 smithi192 ceph-mon[33753]: pgmap v505: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:52.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:51 smithi122 ceph-mon[29744]: pgmap v505: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T17:59:54.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:53 smithi149 ceph-mon[33409]: pgmap v506: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:54.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:53 smithi192 ceph-mon[33753]: pgmap v506: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:54.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:53 smithi122 ceph-mon[29744]: pgmap v506: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T17:59:55.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:59:55.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:59:55.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:59:55.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:54 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:59:55.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:59:55.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:59:55.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:59:55.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:54 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:59:55.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T17:59:55.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T17:59:55.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T17:59:55.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:54 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T17:59:56.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:56 smithi122 ceph-mon[29744]: pgmap v507: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:56.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:56 smithi149 ceph-mon[33409]: pgmap v507: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:56.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:56 smithi192 ceph-mon[33753]: pgmap v507: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T17:59:58.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 17:59:58 smithi122 ceph-mon[29744]: pgmap v508: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:58.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 17:59:58 smithi149 ceph-mon[33409]: pgmap v508: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T17:59:58.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 17:59:58 smithi192 ceph-mon[33753]: pgmap v508: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:00.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:00 smithi122 ceph-mon[29744]: pgmap v509: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:00.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:00 smithi122 ceph-mon[29744]: overall HEALTH_OK 2024-05-03T18:00:00.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:00 smithi149 ceph-mon[33409]: pgmap v509: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:00.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:00 smithi149 ceph-mon[33409]: overall HEALTH_OK 2024-05-03T18:00:00.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:00 smithi192 ceph-mon[33753]: pgmap v509: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:00.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:00 smithi192 ceph-mon[33753]: overall HEALTH_OK 2024-05-03T18:00:02.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:02 smithi149 ceph-mon[33409]: pgmap v510: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:02.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:02 smithi192 ceph-mon[33753]: pgmap v510: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:02.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:02 smithi122 ceph-mon[29744]: pgmap v510: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:04.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:04 smithi149 ceph-mon[33409]: pgmap v511: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:04.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:04 smithi192 ceph-mon[33753]: pgmap v511: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:04.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:04 smithi122 ceph-mon[29744]: pgmap v511: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:06.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:06 smithi149 ceph-mon[33409]: pgmap v512: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:06.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:06 smithi192 ceph-mon[33753]: pgmap v512: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:06.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:06 smithi122 ceph-mon[29744]: pgmap v512: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:08.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:08 smithi149 ceph-mon[33409]: pgmap v513: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:08.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:08 smithi192 ceph-mon[33753]: pgmap v513: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:08.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:08 smithi122 ceph-mon[29744]: pgmap v513: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:10.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:10 smithi149 ceph-mon[33409]: pgmap v514: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:10.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:10 smithi192 ceph-mon[33753]: pgmap v514: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:10.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:10 smithi122 ceph-mon[29744]: pgmap v514: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:12.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:12 smithi149 ceph-mon[33409]: pgmap v515: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:12.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:12 smithi192 ceph-mon[33753]: pgmap v515: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:12.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:12 smithi122 ceph-mon[29744]: pgmap v515: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:14.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:14 smithi149 ceph-mon[33409]: pgmap v516: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:14.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:14 smithi192 ceph-mon[33753]: pgmap v516: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:14.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:14 smithi122 ceph-mon[29744]: pgmap v516: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:16.760 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:16 smithi122 ceph-mon[29744]: pgmap v517: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:16.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:16 smithi149 ceph-mon[33409]: pgmap v517: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:16.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:16 smithi192 ceph-mon[33753]: pgmap v517: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:18.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:18 smithi149 ceph-mon[33409]: pgmap v518: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:18.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:18 smithi192 ceph-mon[33753]: pgmap v518: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:18.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:18 smithi122 ceph-mon[29744]: pgmap v518: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:20.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:20 smithi149 ceph-mon[33409]: pgmap v519: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:20.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:20 smithi192 ceph-mon[33753]: pgmap v519: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:20.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:20 smithi122 ceph-mon[29744]: pgmap v519: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:22.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:22 smithi149 ceph-mon[33409]: pgmap v520: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:22.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:22 smithi192 ceph-mon[33753]: pgmap v520: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:22.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:22 smithi122 ceph-mon[29744]: pgmap v520: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:24.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:24 smithi149 ceph-mon[33409]: pgmap v521: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-03T18:00:24.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:24 smithi192 ceph-mon[33753]: pgmap v521: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-03T18:00:24.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:24 smithi122 ceph-mon[29744]: pgmap v521: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-03T18:00:26.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:26 smithi149 ceph-mon[33409]: pgmap v522: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T18:00:26.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:26 smithi192 ceph-mon[33753]: pgmap v522: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T18:00:26.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:26 smithi122 ceph-mon[29744]: pgmap v522: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T18:00:28.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:28 smithi149 ceph-mon[33409]: pgmap v523: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:00:28.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:28 smithi192 ceph-mon[33753]: pgmap v523: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:00:28.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:28 smithi122 ceph-mon[29744]: pgmap v523: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:00:30.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:30 smithi149 ceph-mon[33409]: pgmap v524: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-03T18:00:30.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:30 smithi192 ceph-mon[33753]: pgmap v524: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-03T18:00:30.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:30 smithi122 ceph-mon[29744]: pgmap v524: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-03T18:00:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:32 smithi149 ceph-mon[33409]: pgmap v525: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-05-03T18:00:32.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:32 smithi192 ceph-mon[33753]: pgmap v525: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-05-03T18:00:32.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:32 smithi122 ceph-mon[29744]: pgmap v525: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-05-03T18:00:34.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:34 smithi149 ceph-mon[33409]: pgmap v526: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:00:34.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:34 smithi192 ceph-mon[33753]: pgmap v526: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:00:34.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:34 smithi122 ceph-mon[29744]: pgmap v526: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:00:36.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:36 smithi149 ceph-mon[33409]: pgmap v527: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:36.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:36 smithi192 ceph-mon[33753]: pgmap v527: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:36.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:36 smithi122 ceph-mon[29744]: pgmap v527: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:38.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:38 smithi149 ceph-mon[33409]: pgmap v528: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:38.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:38 smithi192 ceph-mon[33753]: pgmap v528: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:38.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:38 smithi122 ceph-mon[29744]: pgmap v528: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:40.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:40 smithi149 ceph-mon[33409]: pgmap v529: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:40.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:40 smithi192 ceph-mon[33753]: pgmap v529: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:40.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:40 smithi122 ceph-mon[29744]: pgmap v529: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:42.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:42 smithi149 ceph-mon[33409]: pgmap v530: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:42.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:42 smithi192 ceph-mon[33753]: pgmap v530: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:42.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:42 smithi122 ceph-mon[29744]: pgmap v530: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:44.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:44 smithi149 ceph-mon[33409]: pgmap v531: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:44.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:44 smithi192 ceph-mon[33753]: pgmap v531: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:44.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:44 smithi122 ceph-mon[29744]: pgmap v531: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:46.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:46 smithi192 ceph-mon[33753]: pgmap v532: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:46.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:46 smithi122 ceph-mon[29744]: pgmap v532: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:47.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:46 smithi149 ceph-mon[33409]: pgmap v532: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:48.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:48 smithi192 ceph-mon[33753]: pgmap v533: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:48.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:48 smithi122 ceph-mon[29744]: pgmap v533: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:49.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:48 smithi149 ceph-mon[33409]: pgmap v533: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:50.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:50 smithi192 ceph-mon[33753]: pgmap v534: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:50.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:50 smithi122 ceph-mon[29744]: pgmap v534: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:51.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:50 smithi149 ceph-mon[33409]: pgmap v534: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:00:52.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:52 smithi192 ceph-mon[33753]: pgmap v535: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:52.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:52 smithi122 ceph-mon[29744]: pgmap v535: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:53.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:52 smithi149 ceph-mon[33409]: pgmap v535: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:00:54.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:54 smithi192 ceph-mon[33753]: pgmap v536: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:54.958 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:54 smithi122 ceph-mon[29744]: pgmap v536: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:55.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:54 smithi149 ceph-mon[33409]: pgmap v536: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:00:55.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:55 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:00:55.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:55 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:55.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:55 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:55.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:55 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:55.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:55 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:55.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:55 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:00:55.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:55 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:00:55.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:55 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:00:55.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:00:55.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:55.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:55.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:55.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:55.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:00:55.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:00:55.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:00:56.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:55 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:00:56.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:55 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:56.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:55 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:56.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:55 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:56.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:55 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:00:56.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:55 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:00:56.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:55 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:00:56.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:55 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:00:56.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:56 smithi192 ceph-mon[33753]: pgmap v537: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:56.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:56 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-05-03T18:00:56.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:56 smithi192 ceph-mon[33753]: Unable to set osd_memory_target on smithi122 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-03T18:00:56.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:56 smithi122 ceph-mon[29744]: pgmap v537: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:56.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:56 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-05-03T18:00:56.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:56 smithi122 ceph-mon[29744]: Unable to set osd_memory_target on smithi122 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-03T18:00:57.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:56 smithi149 ceph-mon[33409]: pgmap v537: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:00:57.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:56 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-05-03T18:00:57.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:56 smithi149 ceph-mon[33409]: Unable to set osd_memory_target on smithi122 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-03T18:00:58.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:00:58 smithi192 ceph-mon[33753]: pgmap v538: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:58.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:00:58 smithi122 ceph-mon[29744]: pgmap v538: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:00:59.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:00:58 smithi149 ceph-mon[33409]: pgmap v538: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:00.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:00 smithi192 ceph-mon[33753]: pgmap v539: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:00.958 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:00 smithi122 ceph-mon[29744]: pgmap v539: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:01.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:00 smithi149 ceph-mon[33409]: pgmap v539: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:02.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:02 smithi192 ceph-mon[33753]: pgmap v540: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:02.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:02 smithi122 ceph-mon[29744]: pgmap v540: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:03.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:02 smithi149 ceph-mon[33409]: pgmap v540: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:04.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:04 smithi192 ceph-mon[33753]: pgmap v541: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:04.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:04 smithi122 ceph-mon[29744]: pgmap v541: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:05.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:04 smithi149 ceph-mon[33409]: pgmap v541: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:06.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:06 smithi192 ceph-mon[33753]: pgmap v542: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:06.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:06 smithi122 ceph-mon[29744]: pgmap v542: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:07.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:06 smithi149 ceph-mon[33409]: pgmap v542: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:08.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:08 smithi192 ceph-mon[33753]: pgmap v543: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:08.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:08 smithi122 ceph-mon[29744]: pgmap v543: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:09.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:08 smithi149 ceph-mon[33409]: pgmap v543: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:10.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:10 smithi192 ceph-mon[33753]: pgmap v544: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:10.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:10 smithi122 ceph-mon[29744]: pgmap v544: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:11.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:10 smithi149 ceph-mon[33409]: pgmap v544: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:12.838 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:12 smithi149 ceph-mon[33409]: pgmap v545: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:12.864 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:12 smithi192 ceph-mon[33753]: pgmap v545: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:12.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:12 smithi122 ceph-mon[29744]: pgmap v545: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:14.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:14 smithi192 ceph-mon[33753]: pgmap v546: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:14.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:14 smithi122 ceph-mon[29744]: pgmap v546: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:15.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:14 smithi149 ceph-mon[33409]: pgmap v546: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:16.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:16 smithi192 ceph-mon[33753]: pgmap v547: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:16.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:16 smithi122 ceph-mon[29744]: pgmap v547: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:17.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:16 smithi149 ceph-mon[33409]: pgmap v547: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:18.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:18 smithi192 ceph-mon[33753]: pgmap v548: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:18.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:18 smithi122 ceph-mon[29744]: pgmap v548: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:19.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:18 smithi149 ceph-mon[33409]: pgmap v548: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:20.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:20 smithi192 ceph-mon[33753]: pgmap v549: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:20.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:20 smithi122 ceph-mon[29744]: pgmap v549: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:21.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:20 smithi149 ceph-mon[33409]: pgmap v549: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:21.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:21 smithi192 ceph-mon[33753]: pgmap v550: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:21.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:21 smithi122 ceph-mon[29744]: pgmap v550: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:22.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:21 smithi149 ceph-mon[33409]: pgmap v550: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:24.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:23 smithi149 ceph-mon[33409]: pgmap v551: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:24.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:23 smithi192 ceph-mon[33753]: pgmap v551: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:24.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:23 smithi122 ceph-mon[29744]: pgmap v551: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:26.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:25 smithi149 ceph-mon[33409]: pgmap v552: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:26.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:25 smithi192 ceph-mon[33753]: pgmap v552: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:26.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:25 smithi122 ceph-mon[29744]: pgmap v552: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:28.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:27 smithi149 ceph-mon[33409]: pgmap v553: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:28.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:27 smithi192 ceph-mon[33753]: pgmap v553: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:28.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:27 smithi122 ceph-mon[29744]: pgmap v553: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:30.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:29 smithi149 ceph-mon[33409]: pgmap v554: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:30.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:29 smithi192 ceph-mon[33753]: pgmap v554: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:30.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:29 smithi122 ceph-mon[29744]: pgmap v554: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:32.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:31 smithi149 ceph-mon[33409]: pgmap v555: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:32.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:31 smithi192 ceph-mon[33753]: pgmap v555: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:32.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:31 smithi122 ceph-mon[29744]: pgmap v555: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:34.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:33 smithi149 ceph-mon[33409]: pgmap v556: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:34.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:33 smithi192 ceph-mon[33753]: pgmap v556: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:34.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:33 smithi122 ceph-mon[29744]: pgmap v556: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:36.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:35 smithi149 ceph-mon[33409]: pgmap v557: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:36.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:35 smithi192 ceph-mon[33753]: pgmap v557: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:36.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:35 smithi122 ceph-mon[29744]: pgmap v557: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:38.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:37 smithi149 ceph-mon[33409]: pgmap v558: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:38.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:37 smithi192 ceph-mon[33753]: pgmap v558: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:38.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:37 smithi122 ceph-mon[29744]: pgmap v558: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:40.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:39 smithi149 ceph-mon[33409]: pgmap v559: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:40.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:39 smithi192 ceph-mon[33753]: pgmap v559: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:40.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:39 smithi122 ceph-mon[29744]: pgmap v559: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:42.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:41 smithi149 ceph-mon[33409]: pgmap v560: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:42.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:41 smithi192 ceph-mon[33753]: pgmap v560: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:42.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:41 smithi122 ceph-mon[29744]: pgmap v560: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:44.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:43 smithi149 ceph-mon[33409]: pgmap v561: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:44.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:43 smithi192 ceph-mon[33753]: pgmap v561: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:44.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:43 smithi122 ceph-mon[29744]: pgmap v561: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:46.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:45 smithi149 ceph-mon[33409]: pgmap v562: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:46.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:45 smithi192 ceph-mon[33753]: pgmap v562: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:46.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:45 smithi122 ceph-mon[29744]: pgmap v562: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:48.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:47 smithi149 ceph-mon[33409]: pgmap v563: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:48.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:47 smithi192 ceph-mon[33753]: pgmap v563: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:48.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:47 smithi122 ceph-mon[29744]: pgmap v563: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:50.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:49 smithi149 ceph-mon[33409]: pgmap v564: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:50.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:49 smithi192 ceph-mon[33753]: pgmap v564: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:50.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:49 smithi122 ceph-mon[29744]: pgmap v564: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:01:52.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:51 smithi149 ceph-mon[33409]: pgmap v565: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:52.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:51 smithi192 ceph-mon[33753]: pgmap v565: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:52.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:51 smithi122 ceph-mon[29744]: pgmap v565: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:01:54.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:53 smithi149 ceph-mon[33409]: pgmap v566: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:54.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:53 smithi192 ceph-mon[33753]: pgmap v566: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:54.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:53 smithi122 ceph-mon[29744]: pgmap v566: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:01:56.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:55 smithi149 ceph-mon[33409]: pgmap v567: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:56.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:55 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:01:56.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:55 smithi192 ceph-mon[33753]: pgmap v567: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:56.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:55 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:01:56.209 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:55 smithi122 ceph-mon[29744]: pgmap v567: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:01:56.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:55 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:01:56.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:01:56.684+0000 7fbf367fc640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fbf24005850 con 0x7fbf401034c0 2024-05-03T18:01:56.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:01:56.685+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 8 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fbf24005850 con 0x7fbf401034c0 2024-05-03T18:01:57.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-05-03T18:01:57.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.037 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:56 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi192 to 176.2M 2024-05-03T18:01:57.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:56 smithi149 ceph-mon[33409]: Unable to set osd_memory_target on smithi192 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-03T18:01:57.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:01:57.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:01:57.038 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:56 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:01:57.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.115 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-05-03T18:01:57.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:56 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi192 to 176.2M 2024-05-03T18:01:57.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:56 smithi192 ceph-mon[33753]: Unable to set osd_memory_target on smithi192 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-03T18:01:57.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:01:57.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:01:57.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:56 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:01:57.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-05-03T18:01:57.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:01:57.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:56 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi192 to 176.2M 2024-05-03T18:01:57.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:56 smithi122 ceph-mon[29744]: Unable to set osd_memory_target on smithi192 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-05-03T18:01:57.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:01:57.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:01:57.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:56 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:01:58.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:01:58 smithi149 ceph-mon[33409]: pgmap v568: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:58.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:01:58 smithi192 ceph-mon[33753]: pgmap v568: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:01:58.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:01:58 smithi122 ceph-mon[29744]: pgmap v568: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:00.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:00 smithi149 ceph-mon[33409]: pgmap v569: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:00.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:00 smithi192 ceph-mon[33753]: pgmap v569: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:00.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:00 smithi122 ceph-mon[29744]: pgmap v569: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:02.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:02 smithi149 ceph-mon[33409]: pgmap v570: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:02.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:02 smithi192 ceph-mon[33753]: pgmap v570: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:02.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:02 smithi122 ceph-mon[29744]: pgmap v570: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:04.286 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:04 smithi149 ceph-mon[33409]: pgmap v571: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:04.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:04 smithi192 ceph-mon[33753]: pgmap v571: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:04.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:04 smithi122 ceph-mon[29744]: pgmap v571: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:06.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:06 smithi192 ceph-mon[33753]: pgmap v572: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:06.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:06 smithi122 ceph-mon[29744]: pgmap v572: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:06.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:06 smithi149 ceph-mon[33409]: pgmap v572: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:08.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:08 smithi192 ceph-mon[33753]: pgmap v573: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:08.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:08 smithi122 ceph-mon[29744]: pgmap v573: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:08.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:08 smithi149 ceph-mon[33409]: pgmap v573: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:10.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:10 smithi192 ceph-mon[33753]: pgmap v574: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:10.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:10 smithi122 ceph-mon[29744]: pgmap v574: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:10.536 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:10 smithi149 ceph-mon[33409]: pgmap v574: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:12.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:12 smithi192 ceph-mon[33753]: pgmap v575: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:12.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:12 smithi122 ceph-mon[29744]: pgmap v575: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:12.536 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:12 smithi149 ceph-mon[33409]: pgmap v575: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:14.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:14 smithi149 ceph-mon[33409]: pgmap v576: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:14.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:14 smithi192 ceph-mon[33753]: pgmap v576: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:14.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:14 smithi122 ceph-mon[29744]: pgmap v576: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:16.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:16 smithi192 ceph-mon[33753]: pgmap v577: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:16.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:16 smithi122 ceph-mon[29744]: pgmap v577: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:16.536 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:16 smithi149 ceph-mon[33409]: pgmap v577: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:18.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:18 smithi192 ceph-mon[33753]: pgmap v578: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:18.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:18 smithi122 ceph-mon[29744]: pgmap v578: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:18.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:18 smithi149 ceph-mon[33409]: pgmap v578: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:20.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:20 smithi192 ceph-mon[33753]: pgmap v579: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:20.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:20 smithi122 ceph-mon[29744]: pgmap v579: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:20.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:20 smithi149 ceph-mon[33409]: pgmap v579: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:22.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:22 smithi192 ceph-mon[33753]: pgmap v580: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:22.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:22 smithi122 ceph-mon[29744]: pgmap v580: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:22.536 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:22 smithi149 ceph-mon[33409]: pgmap v580: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:24.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:24 smithi192 ceph-mon[33753]: pgmap v581: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:24.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:24 smithi122 ceph-mon[29744]: pgmap v581: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:24.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:24 smithi149 ceph-mon[33409]: pgmap v581: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:26.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:26 smithi192 ceph-mon[33753]: pgmap v582: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:26.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:26 smithi122 ceph-mon[29744]: pgmap v582: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:26.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:26 smithi149 ceph-mon[33409]: pgmap v582: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:28.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:28 smithi192 ceph-mon[33753]: pgmap v583: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:28.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:28 smithi122 ceph-mon[29744]: pgmap v583: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:28.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:28 smithi149 ceph-mon[33409]: pgmap v583: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:30.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:30 smithi192 ceph-mon[33753]: pgmap v584: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:30.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:30 smithi122 ceph-mon[29744]: pgmap v584: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:30.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:30 smithi149 ceph-mon[33409]: pgmap v584: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:32.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:32 smithi192 ceph-mon[33753]: pgmap v585: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-03T18:02:32.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:32 smithi122 ceph-mon[29744]: pgmap v585: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-03T18:02:32.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:32 smithi149 ceph-mon[33409]: pgmap v585: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-03T18:02:34.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:34 smithi192 ceph-mon[33753]: pgmap v586: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:02:34.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:34 smithi122 ceph-mon[29744]: pgmap v586: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:02:34.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:34 smithi149 ceph-mon[33409]: pgmap v586: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:02:36.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:36 smithi192 ceph-mon[33753]: pgmap v587: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-03T18:02:36.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:36 smithi122 ceph-mon[29744]: pgmap v587: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-03T18:02:36.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:36 smithi149 ceph-mon[33409]: pgmap v587: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-03T18:02:38.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:38 smithi192 ceph-mon[33753]: pgmap v588: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-03T18:02:38.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:38 smithi122 ceph-mon[29744]: pgmap v588: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-03T18:02:38.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:38 smithi149 ceph-mon[33409]: pgmap v588: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-03T18:02:40.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:40 smithi192 ceph-mon[33753]: pgmap v589: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:40.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:40 smithi122 ceph-mon[29744]: pgmap v589: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:40.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:40 smithi149 ceph-mon[33409]: pgmap v589: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:42.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:42 smithi192 ceph-mon[33753]: pgmap v590: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:42 smithi122 ceph-mon[29744]: pgmap v590: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:42.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:42 smithi149 ceph-mon[33409]: pgmap v590: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:44.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:44 smithi192 ceph-mon[33753]: pgmap v591: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:44.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:44 smithi122 ceph-mon[29744]: pgmap v591: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:44.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:44 smithi149 ceph-mon[33409]: pgmap v591: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:46.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:46 smithi192 ceph-mon[33753]: pgmap v592: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:46.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:46 smithi122 ceph-mon[29744]: pgmap v592: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:46.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:46 smithi149 ceph-mon[33409]: pgmap v592: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:48.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:48 smithi192 ceph-mon[33753]: pgmap v593: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:48.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:48 smithi122 ceph-mon[29744]: pgmap v593: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:48.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:48 smithi149 ceph-mon[33409]: pgmap v593: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:50.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:50 smithi192 ceph-mon[33753]: pgmap v594: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:50.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:50 smithi122 ceph-mon[29744]: pgmap v594: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:50.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:50 smithi149 ceph-mon[33409]: pgmap v594: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:02:52.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:52 smithi192 ceph-mon[33753]: pgmap v595: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:52.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:52 smithi122 ceph-mon[29744]: pgmap v595: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:52.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:52 smithi149 ceph-mon[33409]: pgmap v595: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:02:54.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:54 smithi192 ceph-mon[33753]: pgmap v596: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:54.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:54 smithi122 ceph-mon[29744]: pgmap v596: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:54.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:54 smithi149 ceph-mon[33409]: pgmap v596: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:02:56.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:56 smithi122 ceph-mon[29744]: pgmap v597: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:56.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:56 smithi149 ceph-mon[33409]: pgmap v597: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:56.615 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:56 smithi192 ceph-mon[33753]: pgmap v597: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:02:57.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:02:57.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:02:57.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:02:57.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:02:57.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:02:57.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:02:57.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:02:57.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:02:57.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:02:57.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:02:57.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:02:57.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:02:58.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:02:58 smithi122 ceph-mon[29744]: pgmap v598: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:58.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:02:58 smithi149 ceph-mon[33409]: pgmap v598: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:02:58.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:02:58 smithi192 ceph-mon[33753]: pgmap v598: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:00.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:00 smithi122 ceph-mon[29744]: pgmap v599: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:00.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:00 smithi149 ceph-mon[33409]: pgmap v599: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:00.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:00 smithi192 ceph-mon[33753]: pgmap v599: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:02.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:02 smithi122 ceph-mon[29744]: pgmap v600: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:02.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:02 smithi149 ceph-mon[33409]: pgmap v600: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:02.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:02 smithi192 ceph-mon[33753]: pgmap v600: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:04.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:04 smithi122 ceph-mon[29744]: pgmap v601: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:04.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:04 smithi149 ceph-mon[33409]: pgmap v601: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:04.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:04 smithi192 ceph-mon[33753]: pgmap v601: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:06.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:06 smithi122 ceph-mon[29744]: pgmap v602: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:06.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:06 smithi149 ceph-mon[33409]: pgmap v602: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:06.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:06 smithi192 ceph-mon[33753]: pgmap v602: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:08.418 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:08 smithi192 ceph-mon[33753]: pgmap v603: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:08.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:08 smithi122 ceph-mon[29744]: pgmap v603: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:08.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:08 smithi149 ceph-mon[33409]: pgmap v603: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:10.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:10 smithi122 ceph-mon[29744]: pgmap v604: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:10.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:10 smithi149 ceph-mon[33409]: pgmap v604: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:10.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:10 smithi192 ceph-mon[33753]: pgmap v604: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:12.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:12 smithi122 ceph-mon[29744]: pgmap v605: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:12.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:12 smithi149 ceph-mon[33409]: pgmap v605: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:12.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:12 smithi192 ceph-mon[33753]: pgmap v605: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:14.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:14 smithi122 ceph-mon[29744]: pgmap v606: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:14.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:14 smithi149 ceph-mon[33409]: pgmap v606: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:14.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:14 smithi192 ceph-mon[33753]: pgmap v606: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:16.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:16 smithi122 ceph-mon[29744]: pgmap v607: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:16.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:16 smithi149 ceph-mon[33409]: pgmap v607: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:16.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:16 smithi192 ceph-mon[33753]: pgmap v607: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:18.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:18 smithi122 ceph-mon[29744]: pgmap v608: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:18.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:18 smithi149 ceph-mon[33409]: pgmap v608: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:18.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:18 smithi192 ceph-mon[33753]: pgmap v608: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:20.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:20 smithi122 ceph-mon[29744]: pgmap v609: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:20.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:20 smithi149 ceph-mon[33409]: pgmap v609: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:20.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:20 smithi192 ceph-mon[33753]: pgmap v609: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:22.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:22 smithi122 ceph-mon[29744]: pgmap v610: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:22.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:22 smithi149 ceph-mon[33409]: pgmap v610: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:22.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:22 smithi192 ceph-mon[33753]: pgmap v610: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:24.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:24 smithi122 ceph-mon[29744]: pgmap v611: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:24.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:24 smithi149 ceph-mon[33409]: pgmap v611: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:24.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:24 smithi192 ceph-mon[33753]: pgmap v611: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:26.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:26 smithi122 ceph-mon[29744]: pgmap v612: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:26.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:26 smithi149 ceph-mon[33409]: pgmap v612: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:26.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:26 smithi192 ceph-mon[33753]: pgmap v612: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:28.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:28 smithi122 ceph-mon[29744]: pgmap v613: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:28.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:28 smithi149 ceph-mon[33409]: pgmap v613: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:28.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:28 smithi192 ceph-mon[33753]: pgmap v613: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:30.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:30 smithi122 ceph-mon[29744]: pgmap v614: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:30.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:30 smithi149 ceph-mon[33409]: pgmap v614: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:30.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:30 smithi192 ceph-mon[33753]: pgmap v614: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:32.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:32 smithi122 ceph-mon[29744]: pgmap v615: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:32.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:32 smithi149 ceph-mon[33409]: pgmap v615: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:32.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:32 smithi192 ceph-mon[33753]: pgmap v615: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:34.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:34 smithi122 ceph-mon[29744]: pgmap v616: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:34.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:34 smithi149 ceph-mon[33409]: pgmap v616: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:34 smithi192 ceph-mon[33753]: pgmap v616: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:36.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:36 smithi122 ceph-mon[29744]: pgmap v617: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:36.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:36 smithi149 ceph-mon[33409]: pgmap v617: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:36.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:36 smithi192 ceph-mon[33753]: pgmap v617: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:38.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:38 smithi122 ceph-mon[29744]: pgmap v618: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:38.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:38 smithi149 ceph-mon[33409]: pgmap v618: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:38.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:38 smithi192 ceph-mon[33753]: pgmap v618: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:40.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:40 smithi122 ceph-mon[29744]: pgmap v619: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:40.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:40 smithi149 ceph-mon[33409]: pgmap v619: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:40.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:40 smithi192 ceph-mon[33753]: pgmap v619: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:42.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:42 smithi122 ceph-mon[29744]: pgmap v620: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:42.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:42 smithi149 ceph-mon[33409]: pgmap v620: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:42.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:42 smithi192 ceph-mon[33753]: pgmap v620: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:44.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:44 smithi122 ceph-mon[29744]: pgmap v621: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:44.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:44 smithi149 ceph-mon[33409]: pgmap v621: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:44.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:44 smithi192 ceph-mon[33753]: pgmap v621: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:46.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:46 smithi122 ceph-mon[29744]: pgmap v622: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:46.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:46 smithi149 ceph-mon[33409]: pgmap v622: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:46.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:46 smithi192 ceph-mon[33753]: pgmap v622: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:48.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:48 smithi122 ceph-mon[29744]: pgmap v623: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:48.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:48 smithi149 ceph-mon[33409]: pgmap v623: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:48.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:48 smithi192 ceph-mon[33753]: pgmap v623: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:50.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:50 smithi122 ceph-mon[29744]: pgmap v624: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:50.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:50 smithi149 ceph-mon[33409]: pgmap v624: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:50.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:50 smithi192 ceph-mon[33753]: pgmap v624: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:03:52.460 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:52 smithi122 ceph-mon[29744]: pgmap v625: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:52.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:52 smithi149 ceph-mon[33409]: pgmap v625: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:52.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:52 smithi192 ceph-mon[33753]: pgmap v625: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:03:54.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:54 smithi149 ceph-mon[33409]: pgmap v626: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:54.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:54 smithi192 ceph-mon[33753]: pgmap v626: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:54.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:54 smithi122 ceph-mon[29744]: pgmap v626: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:03:56.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:56 smithi149 ceph-mon[33409]: pgmap v627: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:56.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:56 smithi192 ceph-mon[33753]: pgmap v627: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:56.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:56 smithi122 ceph-mon[29744]: pgmap v627: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:03:57.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:57 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:03:57.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:57 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:03:57.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:57 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:03:58.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:58 smithi149 ceph-mon[33409]: pgmap v628: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:58.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:58 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:03:58.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:58 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:03:58.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:03:58 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:03:58.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:58 smithi192 ceph-mon[33753]: pgmap v628: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:58.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:58 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:03:58.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:58 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:03:58.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:03:58 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:03:58.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:58 smithi122 ceph-mon[29744]: pgmap v628: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:03:58.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:58 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:03:58.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:58 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:03:58.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:03:58 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:04:00.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:00 smithi149 ceph-mon[33409]: pgmap v629: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:00.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:00 smithi192 ceph-mon[33753]: pgmap v629: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:00.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:00 smithi122 ceph-mon[29744]: pgmap v629: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:02.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:02 smithi149 ceph-mon[33409]: pgmap v630: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:02.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:02 smithi192 ceph-mon[33753]: pgmap v630: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:02.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:02 smithi122 ceph-mon[29744]: pgmap v630: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:04.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:04 smithi149 ceph-mon[33409]: pgmap v631: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:04.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:04 smithi192 ceph-mon[33753]: pgmap v631: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:04.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:04 smithi122 ceph-mon[29744]: pgmap v631: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:06.537 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:06 smithi149 ceph-mon[33409]: pgmap v632: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:06.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:06 smithi192 ceph-mon[33753]: pgmap v632: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:06.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:06 smithi122 ceph-mon[29744]: pgmap v632: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:08.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:08 smithi149 ceph-mon[33409]: pgmap v633: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:08.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:08 smithi192 ceph-mon[33753]: pgmap v633: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:08.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:08 smithi122 ceph-mon[29744]: pgmap v633: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:10.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:10 smithi149 ceph-mon[33409]: pgmap v634: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:10.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:10 smithi192 ceph-mon[33753]: pgmap v634: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:10.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:10 smithi122 ceph-mon[29744]: pgmap v634: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:11.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:04:11.703+0000 7fbf367fc640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fbf24005b70 con 0x7fbf401034c0 2024-05-03T18:04:11.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:04:11.703+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 9 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fbf24005b70 con 0x7fbf401034c0 2024-05-03T18:04:12.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:12 smithi149 ceph-mon[33409]: pgmap v635: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:12.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:12 smithi192 ceph-mon[33753]: pgmap v635: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:12.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:12 smithi122 ceph-mon[29744]: pgmap v635: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:14.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:14 smithi149 ceph-mon[33409]: pgmap v636: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:14.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:14 smithi192 ceph-mon[33753]: pgmap v636: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:14.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:14 smithi122 ceph-mon[29744]: pgmap v636: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:16.512 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:16 smithi122 ceph-mon[29744]: pgmap v637: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:16.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:16 smithi149 ceph-mon[33409]: pgmap v637: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:16.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:16 smithi192 ceph-mon[33753]: pgmap v637: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:18.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:18 smithi149 ceph-mon[33409]: pgmap v638: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:18.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:18 smithi192 ceph-mon[33753]: pgmap v638: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:18.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:18 smithi122 ceph-mon[29744]: pgmap v638: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:20.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:20 smithi149 ceph-mon[33409]: pgmap v639: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:20.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:20 smithi192 ceph-mon[33753]: pgmap v639: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:20.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:20 smithi122 ceph-mon[29744]: pgmap v639: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:22.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:22 smithi149 ceph-mon[33409]: pgmap v640: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:22.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:22 smithi192 ceph-mon[33753]: pgmap v640: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:22.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:22 smithi122 ceph-mon[29744]: pgmap v640: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:24.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:24 smithi149 ceph-mon[33409]: pgmap v641: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:24.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:24 smithi192 ceph-mon[33753]: pgmap v641: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:24.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:24 smithi122 ceph-mon[29744]: pgmap v641: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:26.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:26 smithi149 ceph-mon[33409]: pgmap v642: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:26.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:26 smithi192 ceph-mon[33753]: pgmap v642: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:26.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:26 smithi122 ceph-mon[29744]: pgmap v642: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:28.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:28 smithi149 ceph-mon[33409]: pgmap v643: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:28.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:28 smithi192 ceph-mon[33753]: pgmap v643: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:28.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:28 smithi122 ceph-mon[29744]: pgmap v643: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:30.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:30 smithi149 ceph-mon[33409]: pgmap v644: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:30.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:30 smithi192 ceph-mon[33753]: pgmap v644: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:30.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:30 smithi122 ceph-mon[29744]: pgmap v644: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:32.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:32 smithi149 ceph-mon[33409]: pgmap v645: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:32.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:32 smithi192 ceph-mon[33753]: pgmap v645: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:32.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:32 smithi122 ceph-mon[29744]: pgmap v645: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:34.538 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:34 smithi149 ceph-mon[33409]: pgmap v646: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:34.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:34 smithi192 ceph-mon[33753]: pgmap v646: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:34.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:34 smithi122 ceph-mon[29744]: pgmap v646: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:36.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:36 smithi192 ceph-mon[33753]: pgmap v647: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:36.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:36 smithi122 ceph-mon[29744]: pgmap v647: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:36.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:36 smithi149 ceph-mon[33409]: pgmap v647: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:38.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:38 smithi192 ceph-mon[33753]: pgmap v648: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:38.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:38 smithi122 ceph-mon[29744]: pgmap v648: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:38.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:38 smithi149 ceph-mon[33409]: pgmap v648: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:40.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:40 smithi192 ceph-mon[33753]: pgmap v649: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:40.710 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:40 smithi122 ceph-mon[29744]: pgmap v649: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:40.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:40 smithi149 ceph-mon[33409]: pgmap v649: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:42.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:42 smithi192 ceph-mon[33753]: pgmap v650: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:42.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:42 smithi122 ceph-mon[29744]: pgmap v650: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:42.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:42 smithi149 ceph-mon[33409]: pgmap v650: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:44.590 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:44 smithi149 ceph-mon[33409]: pgmap v651: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:44.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:44 smithi192 ceph-mon[33753]: pgmap v651: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:44.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:44 smithi122 ceph-mon[29744]: pgmap v651: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:46.616 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:46 smithi192 ceph-mon[33753]: pgmap v652: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:46.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:46 smithi122 ceph-mon[29744]: pgmap v652: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:46.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:46 smithi149 ceph-mon[33409]: pgmap v652: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:48.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:48 smithi192 ceph-mon[33753]: pgmap v653: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:48.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:48 smithi122 ceph-mon[29744]: pgmap v653: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:48.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:48 smithi149 ceph-mon[33409]: pgmap v653: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:50.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:50 smithi192 ceph-mon[33753]: pgmap v654: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:50.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:50 smithi122 ceph-mon[29744]: pgmap v654: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:50.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:50 smithi149 ceph-mon[33409]: pgmap v654: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:04:52.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:52 smithi192 ceph-mon[33753]: pgmap v655: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:52.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:52 smithi122 ceph-mon[29744]: pgmap v655: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:52.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:52 smithi149 ceph-mon[33409]: pgmap v655: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:04:54.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:54 smithi192 ceph-mon[33753]: pgmap v656: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:54.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:54 smithi122 ceph-mon[29744]: pgmap v656: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:54.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:54 smithi149 ceph-mon[33409]: pgmap v656: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:04:56.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:56 smithi192 ceph-mon[33753]: pgmap v657: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:56.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:04:56.709+0000 7fbf367fc640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fbf24005ed0 con 0x7fbf401034c0 2024-05-03T18:04:56.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:04:56.709+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 10 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fbf24005ed0 con 0x7fbf401034c0 2024-05-03T18:04:56.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:56 smithi122 ceph-mon[29744]: pgmap v657: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:56.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:56 smithi149 ceph-mon[33409]: pgmap v657: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:04:58.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:58 smithi192 ceph-mon[33753]: pgmap v658: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:58.617 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:04:58 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:04:58.667 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:58 smithi149 ceph-mon[33409]: pgmap v658: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:58.668 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:04:58 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:04:58.701 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:58 smithi122 ceph-mon[29744]: pgmap v658: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:04:58.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:04:58 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:05:00.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:00 smithi122 ceph-mon[29744]: pgmap v659: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:00.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:05:00.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:05:00.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:00 smithi149 ceph-mon[33409]: pgmap v659: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:00.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:05:00.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:05:00.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:00 smithi192 ceph-mon[33753]: pgmap v659: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:00.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:00.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:05:00.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:05:00.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:05:02.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:02 smithi122 ceph-mon[29744]: pgmap v660: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:02.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:02 smithi149 ceph-mon[33409]: pgmap v660: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:02.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:02 smithi192 ceph-mon[33753]: pgmap v660: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:04.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:04 smithi122 ceph-mon[29744]: pgmap v661: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:04.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:04 smithi149 ceph-mon[33409]: pgmap v661: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:04.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:04 smithi192 ceph-mon[33753]: pgmap v661: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:06.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:06 smithi122 ceph-mon[29744]: pgmap v662: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:06.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:06 smithi149 ceph-mon[33409]: pgmap v662: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:06.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:06 smithi192 ceph-mon[33753]: pgmap v662: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:08.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:08 smithi122 ceph-mon[29744]: pgmap v663: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:08.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:08 smithi149 ceph-mon[33409]: pgmap v663: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:08.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:08 smithi192 ceph-mon[33753]: pgmap v663: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:10.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:10 smithi122 ceph-mon[29744]: pgmap v664: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:10.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:10 smithi149 ceph-mon[33409]: pgmap v664: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:10.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:10 smithi192 ceph-mon[33753]: pgmap v664: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:12.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:12 smithi122 ceph-mon[29744]: pgmap v665: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:12.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:12 smithi149 ceph-mon[33409]: pgmap v665: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:12.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:12 smithi192 ceph-mon[33753]: pgmap v665: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:14.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:14 smithi122 ceph-mon[29744]: pgmap v666: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:14.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:14 smithi149 ceph-mon[33409]: pgmap v666: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:14.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:14 smithi192 ceph-mon[33753]: pgmap v666: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:16.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:16 smithi122 ceph-mon[29744]: pgmap v667: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:16.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:16 smithi149 ceph-mon[33409]: pgmap v667: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:16.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:16 smithi192 ceph-mon[33753]: pgmap v667: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:18.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:18 smithi122 ceph-mon[29744]: pgmap v668: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:18.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:18 smithi149 ceph-mon[33409]: pgmap v668: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:18.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:18 smithi192 ceph-mon[33753]: pgmap v668: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:20.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:20 smithi122 ceph-mon[29744]: pgmap v669: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:20.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:20 smithi149 ceph-mon[33409]: pgmap v669: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:20.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:20 smithi192 ceph-mon[33753]: pgmap v669: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:22.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:22 smithi122 ceph-mon[29744]: pgmap v670: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:22.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:22 smithi149 ceph-mon[33409]: pgmap v670: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:22.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:22 smithi192 ceph-mon[33753]: pgmap v670: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:24.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:24 smithi122 ceph-mon[29744]: pgmap v671: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:24.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:24 smithi149 ceph-mon[33409]: pgmap v671: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:24.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:24 smithi192 ceph-mon[33753]: pgmap v671: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:26.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:26 smithi122 ceph-mon[29744]: pgmap v672: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:26.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:26 smithi149 ceph-mon[33409]: pgmap v672: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:26.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:26 smithi192 ceph-mon[33753]: pgmap v672: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:28.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:28 smithi122 ceph-mon[29744]: pgmap v673: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:28.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:28 smithi149 ceph-mon[33409]: pgmap v673: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:28.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:28 smithi192 ceph-mon[33753]: pgmap v673: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:30.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:30 smithi149 ceph-mon[33409]: pgmap v674: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:30.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:30 smithi192 ceph-mon[33753]: pgmap v674: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:30.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:30 smithi122 ceph-mon[29744]: pgmap v674: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:32 smithi149 ceph-mon[33409]: pgmap v675: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:32.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:32 smithi192 ceph-mon[33753]: pgmap v675: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:32.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:32 smithi122 ceph-mon[29744]: pgmap v675: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:34.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:34 smithi149 ceph-mon[33409]: pgmap v676: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:34.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:34 smithi192 ceph-mon[33753]: pgmap v676: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:34.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:34 smithi122 ceph-mon[29744]: pgmap v676: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:36.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:36 smithi149 ceph-mon[33409]: pgmap v677: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:36.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:36 smithi192 ceph-mon[33753]: pgmap v677: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:36.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:36 smithi122 ceph-mon[29744]: pgmap v677: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:38.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:38 smithi149 ceph-mon[33409]: pgmap v678: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:38.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:38 smithi192 ceph-mon[33753]: pgmap v678: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:38.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:38 smithi122 ceph-mon[29744]: pgmap v678: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:40.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:40 smithi149 ceph-mon[33409]: pgmap v679: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:40.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:40 smithi192 ceph-mon[33753]: pgmap v679: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:40.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:40 smithi122 ceph-mon[29744]: pgmap v679: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:42.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:42 smithi149 ceph-mon[33409]: pgmap v680: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:42.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:42 smithi192 ceph-mon[33753]: pgmap v680: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:42.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:42 smithi122 ceph-mon[29744]: pgmap v680: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:44.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:44 smithi149 ceph-mon[33409]: pgmap v681: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:44.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:44 smithi192 ceph-mon[33753]: pgmap v681: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:44.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:44 smithi122 ceph-mon[29744]: pgmap v681: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:46.763 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:46 smithi122 ceph-mon[29744]: pgmap v682: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:46.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:46 smithi149 ceph-mon[33409]: pgmap v682: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:46.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:46 smithi192 ceph-mon[33753]: pgmap v682: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:48.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:48 smithi149 ceph-mon[33409]: pgmap v683: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:48.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:48 smithi192 ceph-mon[33753]: pgmap v683: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:48.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:48 smithi122 ceph-mon[29744]: pgmap v683: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:50.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:50 smithi149 ceph-mon[33409]: pgmap v684: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:50.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:50 smithi192 ceph-mon[33753]: pgmap v684: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:50.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:50 smithi122 ceph-mon[29744]: pgmap v684: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:05:52.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:52 smithi149 ceph-mon[33409]: pgmap v685: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:52.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:52 smithi192 ceph-mon[33753]: pgmap v685: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:52.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:52 smithi122 ceph-mon[29744]: pgmap v685: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:05:54.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:54 smithi149 ceph-mon[33409]: pgmap v686: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:54.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:54 smithi192 ceph-mon[33753]: pgmap v686: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:54.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:54 smithi122 ceph-mon[29744]: pgmap v686: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:05:56.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:56 smithi149 ceph-mon[33409]: pgmap v687: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:56.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:56 smithi192 ceph-mon[33753]: pgmap v687: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:56.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:56 smithi122 ceph-mon[29744]: pgmap v687: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:05:58.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:05:58 smithi149 ceph-mon[33409]: pgmap v688: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:58.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:05:58 smithi192 ceph-mon[33753]: pgmap v688: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:05:58.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:05:58 smithi122 ceph-mon[29744]: pgmap v688: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:00.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:00 smithi149 ceph-mon[33409]: pgmap v689: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:00.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:00 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:06:00.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:00 smithi192 ceph-mon[33753]: pgmap v689: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:00.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:00 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:06:00.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:00 smithi122 ceph-mon[29744]: pgmap v689: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:00.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:00 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:06:01.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:01 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:01.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:01 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:01.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:01 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:01.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:01 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:01.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:01 smithi122 ceph-mon[29744]: pgmap v690: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:01.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:01 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:06:01.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:01 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:06:01.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:01 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:02.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:01 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:02.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:01 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:02.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:01 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:02.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:01 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:02.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:01 smithi149 ceph-mon[33409]: pgmap v690: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:02.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:01 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:06:02.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:01 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:06:02.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:01 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:02.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:02.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:02.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:02.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:02.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:01 smithi192 ceph-mon[33753]: pgmap v690: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:02.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:06:02.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:06:02.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:01 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:06:04.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:03 smithi192 ceph-mon[33753]: pgmap v691: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:04.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:03 smithi122 ceph-mon[29744]: pgmap v691: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:04.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:03 smithi149 ceph-mon[33409]: pgmap v691: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:06.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:05 smithi192 ceph-mon[33753]: pgmap v692: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:06.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:05 smithi122 ceph-mon[29744]: pgmap v692: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:06.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:05 smithi149 ceph-mon[33409]: pgmap v692: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:08.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:07 smithi192 ceph-mon[33753]: pgmap v693: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:08.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:07 smithi122 ceph-mon[29744]: pgmap v693: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:08.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:07 smithi149 ceph-mon[33409]: pgmap v693: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:10.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:09 smithi192 ceph-mon[33753]: pgmap v694: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:10.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:09 smithi122 ceph-mon[29744]: pgmap v694: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:10.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:09 smithi149 ceph-mon[33409]: pgmap v694: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:12.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:11 smithi192 ceph-mon[33753]: pgmap v695: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:12.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:11 smithi122 ceph-mon[29744]: pgmap v695: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:12.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:11 smithi149 ceph-mon[33409]: pgmap v695: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:14.091 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:13 smithi149 ceph-mon[33409]: pgmap v696: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:14.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:13 smithi192 ceph-mon[33753]: pgmap v696: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:14.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:13 smithi122 ceph-mon[29744]: pgmap v696: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:16.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:15 smithi192 ceph-mon[33753]: pgmap v697: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:16.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:15 smithi122 ceph-mon[29744]: pgmap v697: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:16.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:15 smithi149 ceph-mon[33409]: pgmap v697: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:18.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:17 smithi192 ceph-mon[33753]: pgmap v698: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:18.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:17 smithi122 ceph-mon[29744]: pgmap v698: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:18.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:17 smithi149 ceph-mon[33409]: pgmap v698: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:20.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:19 smithi122 ceph-mon[29744]: pgmap v699: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:20.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:19 smithi149 ceph-mon[33409]: pgmap v699: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:20.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:19 smithi192 ceph-mon[33753]: pgmap v699: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:22.198 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:21 smithi192 ceph-mon[33753]: pgmap v700: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:22.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:21 smithi122 ceph-mon[29744]: pgmap v700: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:22.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:21 smithi149 ceph-mon[33409]: pgmap v700: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:24.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:23 smithi122 ceph-mon[29744]: pgmap v701: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:24.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:23 smithi149 ceph-mon[33409]: pgmap v701: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:24.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:23 smithi192 ceph-mon[33753]: pgmap v701: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:26.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:25 smithi122 ceph-mon[29744]: pgmap v702: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:26.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:25 smithi149 ceph-mon[33409]: pgmap v702: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:26.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:25 smithi192 ceph-mon[33753]: pgmap v702: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:28.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:27 smithi122 ceph-mon[29744]: pgmap v703: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:28.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:27 smithi149 ceph-mon[33409]: pgmap v703: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:28.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:27 smithi192 ceph-mon[33753]: pgmap v703: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:30.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:29 smithi122 ceph-mon[29744]: pgmap v704: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:30.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:29 smithi149 ceph-mon[33409]: pgmap v704: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:30.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:29 smithi192 ceph-mon[33753]: pgmap v704: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:32.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:31 smithi122 ceph-mon[29744]: pgmap v705: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:32.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:31 smithi149 ceph-mon[33409]: pgmap v705: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:32.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:31 smithi192 ceph-mon[33753]: pgmap v705: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:34.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:33 smithi122 ceph-mon[29744]: pgmap v706: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:34.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:33 smithi149 ceph-mon[33409]: pgmap v706: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:34.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:33 smithi192 ceph-mon[33753]: pgmap v706: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:36.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:35 smithi122 ceph-mon[29744]: pgmap v707: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:36.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:35 smithi149 ceph-mon[33409]: pgmap v707: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:36.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:35 smithi192 ceph-mon[33753]: pgmap v707: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:38.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:37 smithi149 ceph-mon[33409]: pgmap v708: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:38.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:37 smithi192 ceph-mon[33753]: pgmap v708: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:38.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:37 smithi122 ceph-mon[29744]: pgmap v708: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:40.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:39 smithi149 ceph-mon[33409]: pgmap v709: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:40.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:39 smithi192 ceph-mon[33753]: pgmap v709: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:40.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:39 smithi122 ceph-mon[29744]: pgmap v709: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:42.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:41 smithi149 ceph-mon[33409]: pgmap v710: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:42.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:41 smithi192 ceph-mon[33753]: pgmap v710: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:42.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:41 smithi122 ceph-mon[29744]: pgmap v710: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:44.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:43 smithi149 ceph-mon[33409]: pgmap v711: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:44.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:43 smithi192 ceph-mon[33753]: pgmap v711: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:44.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:43 smithi122 ceph-mon[29744]: pgmap v711: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:46.264 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:45 smithi122 ceph-mon[29744]: pgmap v712: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:46.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:45 smithi149 ceph-mon[33409]: pgmap v712: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:46.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:45 smithi192 ceph-mon[33753]: pgmap v712: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:48.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:47 smithi149 ceph-mon[33409]: pgmap v713: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:48.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:47 smithi192 ceph-mon[33753]: pgmap v713: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:48.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:47 smithi122 ceph-mon[29744]: pgmap v713: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:50.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:49 smithi149 ceph-mon[33409]: pgmap v714: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:50.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:49 smithi192 ceph-mon[33753]: pgmap v714: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:50.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:49 smithi122 ceph-mon[29744]: pgmap v714: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:06:52.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:51 smithi149 ceph-mon[33409]: pgmap v715: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:52.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:51 smithi192 ceph-mon[33753]: pgmap v715: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:52.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:51 smithi122 ceph-mon[29744]: pgmap v715: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:06:54.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:53 smithi149 ceph-mon[33409]: pgmap v716: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:54.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:53 smithi192 ceph-mon[33753]: pgmap v716: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:54.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:53 smithi122 ceph-mon[29744]: pgmap v716: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:06:56.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:55 smithi149 ceph-mon[33409]: pgmap v717: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:56.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:55 smithi192 ceph-mon[33753]: pgmap v717: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:56.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:55 smithi122 ceph-mon[29744]: pgmap v717: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:06:58.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:57 smithi149 ceph-mon[33409]: pgmap v718: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:58.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:57 smithi192 ceph-mon[33753]: pgmap v718: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:06:58.461 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:57 smithi122 ceph-mon[29744]: pgmap v718: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:00.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:06:59 smithi149 ceph-mon[33409]: pgmap v719: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:00.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:06:59 smithi192 ceph-mon[33753]: pgmap v719: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:00.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:06:59 smithi122 ceph-mon[29744]: pgmap v719: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:02.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:02 smithi149 ceph-mon[33409]: pgmap v720: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:02.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:02 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:07:02.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:02 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:07:02.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:02 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:07:02.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:02 smithi192 ceph-mon[33753]: pgmap v720: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:02.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:02 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:07:02.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:02 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:07:02.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:02 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:07:02.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:02 smithi122 ceph-mon[29744]: pgmap v720: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:02.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:02 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:07:02.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:02 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:07:02.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:02 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:07:03.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:07:03.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:07:03.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:07:03.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:07:03.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:07:03.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:07:03.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:07:03.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:07:03.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:07:04.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:04 smithi122 ceph-mon[29744]: pgmap v721: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:05.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:04 smithi149 ceph-mon[33409]: pgmap v721: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:05.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:04 smithi192 ceph-mon[33753]: pgmap v721: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:06.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:06 smithi122 ceph-mon[29744]: pgmap v722: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:07.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:06 smithi149 ceph-mon[33409]: pgmap v722: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:07.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:06 smithi192 ceph-mon[33753]: pgmap v722: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:08.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:08 smithi122 ceph-mon[29744]: pgmap v723: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:09.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:08 smithi149 ceph-mon[33409]: pgmap v723: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:09.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:08 smithi192 ceph-mon[33753]: pgmap v723: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:10.919 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:10 smithi192 ceph-mon[33753]: pgmap v724: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:10.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:10 smithi122 ceph-mon[29744]: pgmap v724: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:11.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:10 smithi149 ceph-mon[33409]: pgmap v724: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:12.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:12 smithi122 ceph-mon[29744]: pgmap v725: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:13.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:12 smithi149 ceph-mon[33409]: pgmap v725: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:13.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:12 smithi192 ceph-mon[33753]: pgmap v725: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:13.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:13 smithi122 ceph-mon[29744]: pgmap v726: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:14.039 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:13 smithi149 ceph-mon[33409]: pgmap v726: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:14.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:13 smithi192 ceph-mon[33753]: pgmap v726: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:16.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:15 smithi192 ceph-mon[33753]: pgmap v727: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:16.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:15 smithi122 ceph-mon[29744]: pgmap v727: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:15 smithi149 ceph-mon[33409]: pgmap v727: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:18.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:17 smithi192 ceph-mon[33753]: pgmap v728: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:18.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:17 smithi122 ceph-mon[29744]: pgmap v728: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:18.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:17 smithi149 ceph-mon[33409]: pgmap v728: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:20.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:19 smithi192 ceph-mon[33753]: pgmap v729: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:20.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:19 smithi122 ceph-mon[29744]: pgmap v729: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:20.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:19 smithi149 ceph-mon[33409]: pgmap v729: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:22.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:21 smithi192 ceph-mon[33753]: pgmap v730: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:22.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:21 smithi122 ceph-mon[29744]: pgmap v730: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:22.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:21 smithi149 ceph-mon[33409]: pgmap v730: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:24.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:23 smithi192 ceph-mon[33753]: pgmap v731: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:24.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:23 smithi122 ceph-mon[29744]: pgmap v731: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:24.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:23 smithi149 ceph-mon[33409]: pgmap v731: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:26.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:25 smithi192 ceph-mon[33753]: pgmap v732: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:26.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:25 smithi122 ceph-mon[29744]: pgmap v732: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:26.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:25 smithi149 ceph-mon[33409]: pgmap v732: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:28.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:27 smithi192 ceph-mon[33753]: pgmap v733: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:28.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:27 smithi122 ceph-mon[29744]: pgmap v733: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:28.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:27 smithi149 ceph-mon[33409]: pgmap v733: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:30.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:29 smithi192 ceph-mon[33753]: pgmap v734: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:30.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:29 smithi122 ceph-mon[29744]: pgmap v734: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:30.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:29 smithi149 ceph-mon[33409]: pgmap v734: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:32.117 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:31 smithi192 ceph-mon[33753]: pgmap v735: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:32.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:31 smithi122 ceph-mon[29744]: pgmap v735: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:32.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:31 smithi149 ceph-mon[33409]: pgmap v735: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:34.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:33 smithi122 ceph-mon[29744]: pgmap v736: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:34.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:33 smithi149 ceph-mon[33409]: pgmap v736: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:34.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:33 smithi192 ceph-mon[33753]: pgmap v736: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:36.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:35 smithi122 ceph-mon[29744]: pgmap v737: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:36.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:35 smithi149 ceph-mon[33409]: pgmap v737: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:36.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:35 smithi192 ceph-mon[33753]: pgmap v737: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:38.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:37 smithi122 ceph-mon[29744]: pgmap v738: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:38.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:37 smithi149 ceph-mon[33409]: pgmap v738: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:38.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:37 smithi192 ceph-mon[33753]: pgmap v738: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:40.170 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:39 smithi192 ceph-mon[33753]: pgmap v739: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:40.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:39 smithi122 ceph-mon[29744]: pgmap v739: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:40.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:39 smithi149 ceph-mon[33409]: pgmap v739: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:42.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:41 smithi122 ceph-mon[29744]: pgmap v740: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:42.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:41 smithi149 ceph-mon[33409]: pgmap v740: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:42.369 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:41 smithi192 ceph-mon[33753]: pgmap v740: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:44.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:43 smithi122 ceph-mon[29744]: pgmap v741: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:44.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:43 smithi149 ceph-mon[33409]: pgmap v741: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:44.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:43 smithi192 ceph-mon[33753]: pgmap v741: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:46.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:45 smithi122 ceph-mon[29744]: pgmap v742: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:46.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:45 smithi149 ceph-mon[33409]: pgmap v742: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:46.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:45 smithi192 ceph-mon[33753]: pgmap v742: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:48.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:47 smithi122 ceph-mon[29744]: pgmap v743: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:48.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:47 smithi149 ceph-mon[33409]: pgmap v743: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:48.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:47 smithi192 ceph-mon[33753]: pgmap v743: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:50.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:49 smithi122 ceph-mon[29744]: pgmap v744: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:50.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:49 smithi149 ceph-mon[33409]: pgmap v744: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:50.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:49 smithi192 ceph-mon[33753]: pgmap v744: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:07:52.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:51 smithi122 ceph-mon[29744]: pgmap v745: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:52.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:51 smithi149 ceph-mon[33409]: pgmap v745: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:52.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:51 smithi192 ceph-mon[33753]: pgmap v745: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:07:54.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:53 smithi122 ceph-mon[29744]: pgmap v746: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:54.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:53 smithi149 ceph-mon[33409]: pgmap v746: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:54.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:53 smithi192 ceph-mon[33753]: pgmap v746: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:07:56.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:55 smithi122 ceph-mon[29744]: pgmap v747: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:56.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:55 smithi149 ceph-mon[33409]: pgmap v747: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:56.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:55 smithi192 ceph-mon[33753]: pgmap v747: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:07:56.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:07:56.734+0000 7fbf367fc640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fbf24006110 con 0x7fbf401034c0 2024-05-03T18:07:56.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:07:56.734+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 11 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fbf24006110 con 0x7fbf401034c0 2024-05-03T18:07:58.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:57 smithi122 ceph-mon[29744]: pgmap v748: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:58.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:57 smithi149 ceph-mon[33409]: pgmap v748: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:07:58.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:57 smithi192 ceph-mon[33753]: pgmap v748: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:00.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:07:59 smithi122 ceph-mon[29744]: pgmap v749: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:00.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:07:59 smithi149 ceph-mon[33409]: pgmap v749: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:00.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:07:59 smithi192 ceph-mon[33753]: pgmap v749: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:02.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:01 smithi122 ceph-mon[29744]: pgmap v750: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:02.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:01 smithi149 ceph-mon[33409]: pgmap v750: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:02.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:01 smithi192 ceph-mon[33753]: pgmap v750: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:03.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:02 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:08:03.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:02 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:08:03.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:02 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:08:04.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:03 smithi122 ceph-mon[29744]: pgmap v751: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:04.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:08:04.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:08:04.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:03 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:08:04.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:03 smithi149 ceph-mon[33409]: pgmap v751: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:04.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:08:04.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:08:04.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:03 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:08:04.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:03 smithi192 ceph-mon[33753]: pgmap v751: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:04.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:08:04.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:08:04.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:03 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:08:06.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:05 smithi122 ceph-mon[29744]: pgmap v752: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:06.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:05 smithi149 ceph-mon[33409]: pgmap v752: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:06.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:05 smithi192 ceph-mon[33753]: pgmap v752: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:08.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:07 smithi122 ceph-mon[29744]: pgmap v753: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:08.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:07 smithi149 ceph-mon[33409]: pgmap v753: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:08.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:07 smithi192 ceph-mon[33753]: pgmap v753: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:10.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:09 smithi122 ceph-mon[29744]: pgmap v754: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:10.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:09 smithi149 ceph-mon[33409]: pgmap v754: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:10.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:09 smithi192 ceph-mon[33753]: pgmap v754: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:12.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:11 smithi122 ceph-mon[29744]: pgmap v755: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:12.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:11 smithi149 ceph-mon[33409]: pgmap v755: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:12.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:11 smithi192 ceph-mon[33753]: pgmap v755: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:14.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:13 smithi122 ceph-mon[29744]: pgmap v756: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:14.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:13 smithi149 ceph-mon[33409]: pgmap v756: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:14.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:13 smithi192 ceph-mon[33753]: pgmap v756: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:16.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:15 smithi122 ceph-mon[29744]: pgmap v757: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:16.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:15 smithi149 ceph-mon[33409]: pgmap v757: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:16.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:15 smithi192 ceph-mon[33753]: pgmap v757: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:18.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:17 smithi122 ceph-mon[29744]: pgmap v758: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:18.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:17 smithi149 ceph-mon[33409]: pgmap v758: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:18.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:17 smithi192 ceph-mon[33753]: pgmap v758: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:20.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:19 smithi149 ceph-mon[33409]: pgmap v759: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:20.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:19 smithi192 ceph-mon[33753]: pgmap v759: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:20.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:19 smithi122 ceph-mon[29744]: pgmap v759: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:22.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:21 smithi149 ceph-mon[33409]: pgmap v760: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:22.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:21 smithi192 ceph-mon[33753]: pgmap v760: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:22.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:21 smithi122 ceph-mon[29744]: pgmap v760: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:24.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:23 smithi149 ceph-mon[33409]: pgmap v761: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:24.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:23 smithi192 ceph-mon[33753]: pgmap v761: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:24.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:23 smithi122 ceph-mon[29744]: pgmap v761: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:26.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:25 smithi149 ceph-mon[33409]: pgmap v762: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:26.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:25 smithi192 ceph-mon[33753]: pgmap v762: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:26.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:25 smithi122 ceph-mon[29744]: pgmap v762: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:28.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:27 smithi149 ceph-mon[33409]: pgmap v763: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:28.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:27 smithi192 ceph-mon[33753]: pgmap v763: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:28.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:27 smithi122 ceph-mon[29744]: pgmap v763: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:30.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:29 smithi149 ceph-mon[33409]: pgmap v764: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:30.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:29 smithi192 ceph-mon[33753]: pgmap v764: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:30.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:29 smithi122 ceph-mon[29744]: pgmap v764: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:32.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:31 smithi149 ceph-mon[33409]: pgmap v765: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:32.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:31 smithi192 ceph-mon[33753]: pgmap v765: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:32.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:31 smithi122 ceph-mon[29744]: pgmap v765: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:34.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:33 smithi149 ceph-mon[33409]: pgmap v766: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:34.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:33 smithi192 ceph-mon[33753]: pgmap v766: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:34.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:33 smithi122 ceph-mon[29744]: pgmap v766: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:36.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:35 smithi149 ceph-mon[33409]: pgmap v767: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:36.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:35 smithi192 ceph-mon[33753]: pgmap v767: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:36.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:35 smithi122 ceph-mon[29744]: pgmap v767: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:38.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:37 smithi149 ceph-mon[33409]: pgmap v768: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:38.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:37 smithi192 ceph-mon[33753]: pgmap v768: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:38.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:37 smithi122 ceph-mon[29744]: pgmap v768: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:40.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:39 smithi149 ceph-mon[33409]: pgmap v769: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:40.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:39 smithi192 ceph-mon[33753]: pgmap v769: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:40.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:39 smithi122 ceph-mon[29744]: pgmap v769: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:42.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:41 smithi149 ceph-mon[33409]: pgmap v770: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:42.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:41 smithi192 ceph-mon[33753]: pgmap v770: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:42.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:41 smithi122 ceph-mon[29744]: pgmap v770: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:44.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:44 smithi149 ceph-mon[33409]: pgmap v771: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:44.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:44 smithi192 ceph-mon[33753]: pgmap v771: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:44.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:44 smithi122 ceph-mon[29744]: pgmap v771: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:46.265 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:46 smithi122 ceph-mon[29744]: pgmap v772: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:46.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:46 smithi149 ceph-mon[33409]: pgmap v772: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:46.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:46 smithi192 ceph-mon[33753]: pgmap v772: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:48.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:48 smithi149 ceph-mon[33409]: pgmap v773: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:48.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:48 smithi192 ceph-mon[33753]: pgmap v773: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:48.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:48 smithi122 ceph-mon[29744]: pgmap v773: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:50.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:50 smithi149 ceph-mon[33409]: pgmap v774: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:50.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:50 smithi192 ceph-mon[33753]: pgmap v774: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:50.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:50 smithi122 ceph-mon[29744]: pgmap v774: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:08:52.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:52 smithi149 ceph-mon[33409]: pgmap v775: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:52.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:52 smithi192 ceph-mon[33753]: pgmap v775: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:52.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:52 smithi122 ceph-mon[29744]: pgmap v775: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:08:54.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:54 smithi149 ceph-mon[33409]: pgmap v776: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:54.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:54 smithi192 ceph-mon[33753]: pgmap v776: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:54.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:54 smithi122 ceph-mon[29744]: pgmap v776: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:08:56.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:56 smithi149 ceph-mon[33409]: pgmap v777: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:56.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:56 smithi192 ceph-mon[33753]: pgmap v777: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:56.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:56 smithi122 ceph-mon[29744]: pgmap v777: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:08:58.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:08:58 smithi149 ceph-mon[33409]: pgmap v778: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:58.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:08:58 smithi192 ceph-mon[33753]: pgmap v778: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:08:58.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:08:58 smithi122 ceph-mon[29744]: pgmap v778: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:00.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:00 smithi149 ceph-mon[33409]: pgmap v779: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:00.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:00 smithi192 ceph-mon[33753]: pgmap v779: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:00.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:00 smithi122 ceph-mon[29744]: pgmap v779: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:02.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:02 smithi149 ceph-mon[33409]: pgmap v780: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:02.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:02 smithi192 ceph-mon[33753]: pgmap v780: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:02.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:02 smithi122 ceph-mon[29744]: pgmap v780: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:04.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:04 smithi149 ceph-mon[33409]: pgmap v781: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:04.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:04 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:09:04.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:04 smithi192 ceph-mon[33753]: pgmap v781: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:04.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:04 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:09:04.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:04 smithi122 ceph-mon[29744]: pgmap v781: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:04.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:04 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:09:05.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:05 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:09:05.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:05 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:09:05.290 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:05 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:09:05.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:05 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:09:05.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:05 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:09:05.369 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:05 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:09:05.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:05 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:09:05.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:05 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:09:05.462 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:05 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:09:06.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:06 smithi149 ceph-mon[33409]: pgmap v782: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:06.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:06 smithi192 ceph-mon[33753]: pgmap v782: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:06.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:06 smithi122 ceph-mon[29744]: pgmap v782: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:08.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:08 smithi149 ceph-mon[33409]: pgmap v783: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:08.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:08 smithi192 ceph-mon[33753]: pgmap v783: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:08.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:08 smithi122 ceph-mon[29744]: pgmap v783: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:10.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:10 smithi149 ceph-mon[33409]: pgmap v784: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:10.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:10 smithi192 ceph-mon[33753]: pgmap v784: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:10.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:10 smithi122 ceph-mon[29744]: pgmap v784: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:12.540 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:12 smithi149 ceph-mon[33409]: pgmap v785: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:12.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:12 smithi192 ceph-mon[33753]: pgmap v785: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:12.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:12 smithi122 ceph-mon[29744]: pgmap v785: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:14.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:14 smithi149 ceph-mon[33409]: pgmap v786: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:14.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:14 smithi192 ceph-mon[33753]: pgmap v786: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:14.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:14 smithi122 ceph-mon[29744]: pgmap v786: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:16.515 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:16 smithi122 ceph-mon[29744]: pgmap v787: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:16.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:16 smithi149 ceph-mon[33409]: pgmap v787: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:16.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:16 smithi192 ceph-mon[33753]: pgmap v787: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:18.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:18 smithi149 ceph-mon[33409]: pgmap v788: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:18.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:18 smithi192 ceph-mon[33753]: pgmap v788: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:18.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:18 smithi122 ceph-mon[29744]: pgmap v788: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:20.540 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:20 smithi149 ceph-mon[33409]: pgmap v789: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:20.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:20 smithi192 ceph-mon[33753]: pgmap v789: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:20.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:20 smithi122 ceph-mon[29744]: pgmap v789: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:22.540 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:22 smithi149 ceph-mon[33409]: pgmap v790: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:22.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:22 smithi192 ceph-mon[33753]: pgmap v790: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:22.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:22 smithi122 ceph-mon[29744]: pgmap v790: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:24.540 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:24 smithi149 ceph-mon[33409]: pgmap v791: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:24.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:24 smithi192 ceph-mon[33753]: pgmap v791: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:24.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:24 smithi122 ceph-mon[29744]: pgmap v791: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:26.540 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:26 smithi149 ceph-mon[33409]: pgmap v792: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:26.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:26 smithi192 ceph-mon[33753]: pgmap v792: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:26.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:26 smithi122 ceph-mon[29744]: pgmap v792: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:28.540 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:28 smithi149 ceph-mon[33409]: pgmap v793: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:28.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:28 smithi192 ceph-mon[33753]: pgmap v793: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:28.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:28 smithi122 ceph-mon[29744]: pgmap v793: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:30.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:30 smithi149 ceph-mon[33409]: pgmap v794: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:30.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:30 smithi192 ceph-mon[33753]: pgmap v794: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:30.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:30 smithi122 ceph-mon[29744]: pgmap v794: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:32.539 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:32 smithi149 ceph-mon[33409]: pgmap v795: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:32.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:32 smithi192 ceph-mon[33753]: pgmap v795: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:32.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:32 smithi122 ceph-mon[29744]: pgmap v795: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:34.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:34 smithi192 ceph-mon[33753]: pgmap v796: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:34.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:34 smithi122 ceph-mon[29744]: pgmap v796: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:34.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:34 smithi149 ceph-mon[33409]: pgmap v796: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:36.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:36 smithi192 ceph-mon[33753]: pgmap v797: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:36.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:36 smithi122 ceph-mon[29744]: pgmap v797: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:36.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:36 smithi149 ceph-mon[33409]: pgmap v797: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:38.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:38 smithi192 ceph-mon[33753]: pgmap v798: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:38.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:38 smithi122 ceph-mon[29744]: pgmap v798: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:38.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:38 smithi149 ceph-mon[33409]: pgmap v798: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:40.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:40 smithi192 ceph-mon[33753]: pgmap v799: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:40.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:40 smithi122 ceph-mon[29744]: pgmap v799: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:40.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:40 smithi149 ceph-mon[33409]: pgmap v799: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:42.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:42 smithi192 ceph-mon[33753]: pgmap v800: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:42.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:42 smithi122 ceph-mon[29744]: pgmap v800: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:42.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:42 smithi149 ceph-mon[33409]: pgmap v800: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:44.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:44 smithi192 ceph-mon[33753]: pgmap v801: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:44.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:44 smithi122 ceph-mon[29744]: pgmap v801: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:44.789 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:44 smithi149 ceph-mon[33409]: pgmap v801: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:46.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:46 smithi149 ceph-mon[33409]: pgmap v802: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:46.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:46 smithi192 ceph-mon[33753]: pgmap v802: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:46.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:46 smithi122 ceph-mon[29744]: pgmap v802: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:48.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:48 smithi192 ceph-mon[33753]: pgmap v803: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:48.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:48 smithi122 ceph-mon[29744]: pgmap v803: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:48.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:48 smithi149 ceph-mon[33409]: pgmap v803: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:50.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:50 smithi192 ceph-mon[33753]: pgmap v804: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:50.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:50 smithi122 ceph-mon[29744]: pgmap v804: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:50.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:50 smithi149 ceph-mon[33409]: pgmap v804: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:09:52.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:52 smithi192 ceph-mon[33753]: pgmap v805: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:52.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:52 smithi122 ceph-mon[29744]: pgmap v805: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:52.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:52 smithi149 ceph-mon[33409]: pgmap v805: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:09:54.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:54 smithi192 ceph-mon[33753]: pgmap v806: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:54.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:54 smithi122 ceph-mon[29744]: pgmap v806: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:54.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:54 smithi149 ceph-mon[33409]: pgmap v806: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:09:56.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:56 smithi192 ceph-mon[33753]: pgmap v807: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:56.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:56 smithi122 ceph-mon[29744]: pgmap v807: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:56.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:56 smithi149 ceph-mon[33409]: pgmap v807: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:09:58.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:09:58 smithi192 ceph-mon[33753]: pgmap v808: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:58.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:09:58 smithi122 ceph-mon[29744]: pgmap v808: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:09:58.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:09:58 smithi149 ceph-mon[33409]: pgmap v808: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:00.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:00 smithi192 ceph-mon[33753]: pgmap v809: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:00.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:00 smithi192 ceph-mon[33753]: overall HEALTH_OK 2024-05-03T18:10:00.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:00 smithi122 ceph-mon[29744]: pgmap v809: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:00.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:00 smithi122 ceph-mon[29744]: overall HEALTH_OK 2024-05-03T18:10:00.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:00 smithi149 ceph-mon[33409]: pgmap v809: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:00.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:00 smithi149 ceph-mon[33409]: overall HEALTH_OK 2024-05-03T18:10:02.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:02 smithi192 ceph-mon[33753]: pgmap v810: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:02.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:02 smithi122 ceph-mon[29744]: pgmap v810: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:02.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:02 smithi149 ceph-mon[33409]: pgmap v810: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:04.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:04 smithi192 ceph-mon[33753]: pgmap v811: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:04.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:04 smithi122 ceph-mon[29744]: pgmap v811: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:04.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:04 smithi149 ceph-mon[33409]: pgmap v811: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:05.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:05 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:10:05.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:05 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:10:05.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:05 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:10:05.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:05 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:10:05.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:05 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:10:05.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:05 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:10:05.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:05 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:10:05.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:05 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:10:05.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:05 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:10:05.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:05 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:10:05.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:05 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:10:05.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:05 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:10:06.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:06 smithi192 ceph-mon[33753]: pgmap v812: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:06.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:06 smithi122 ceph-mon[29744]: pgmap v812: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:06.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:06 smithi149 ceph-mon[33409]: pgmap v812: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:08.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:08 smithi192 ceph-mon[33753]: pgmap v813: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:08.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:08 smithi122 ceph-mon[29744]: pgmap v813: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:08.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:08 smithi149 ceph-mon[33409]: pgmap v813: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:10.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:10 smithi192 ceph-mon[33753]: pgmap v814: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:10.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:10 smithi122 ceph-mon[29744]: pgmap v814: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:10.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:10 smithi149 ceph-mon[33409]: pgmap v814: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:12.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:12 smithi192 ceph-mon[33753]: pgmap v815: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:12.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:12 smithi122 ceph-mon[29744]: pgmap v815: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:12.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:12 smithi149 ceph-mon[33409]: pgmap v815: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:14.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:14 smithi192 ceph-mon[33753]: pgmap v816: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:14.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:14 smithi122 ceph-mon[29744]: pgmap v816: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:14.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:14 smithi149 ceph-mon[33409]: pgmap v816: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:16.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:16 smithi192 ceph-mon[33753]: pgmap v817: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:16.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:16 smithi122 ceph-mon[29744]: pgmap v817: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:16.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:16 smithi149 ceph-mon[33409]: pgmap v817: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:18.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:18 smithi192 ceph-mon[33753]: pgmap v818: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:18.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:18 smithi122 ceph-mon[29744]: pgmap v818: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:18.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:18 smithi149 ceph-mon[33409]: pgmap v818: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:20.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:20 smithi122 ceph-mon[29744]: pgmap v819: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:20.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:20 smithi149 ceph-mon[33409]: pgmap v819: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:20.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:20 smithi192 ceph-mon[33753]: pgmap v819: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:22.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:22 smithi122 ceph-mon[29744]: pgmap v820: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:22.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:22 smithi149 ceph-mon[33409]: pgmap v820: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:22.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:22 smithi192 ceph-mon[33753]: pgmap v820: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:24.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:24 smithi122 ceph-mon[29744]: pgmap v821: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:24.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:24 smithi149 ceph-mon[33409]: pgmap v821: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:24.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:24 smithi192 ceph-mon[33753]: pgmap v821: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:26.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:26 smithi122 ceph-mon[29744]: pgmap v822: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:26.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:26 smithi149 ceph-mon[33409]: pgmap v822: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:26.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:26 smithi192 ceph-mon[33753]: pgmap v822: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:28.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:28 smithi122 ceph-mon[29744]: pgmap v823: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:28.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:28 smithi149 ceph-mon[33409]: pgmap v823: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:28.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:28 smithi192 ceph-mon[33753]: pgmap v823: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:30.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:30 smithi122 ceph-mon[29744]: pgmap v824: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:30.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:30 smithi149 ceph-mon[33409]: pgmap v824: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:30.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:30 smithi192 ceph-mon[33753]: pgmap v824: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:32.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:32 smithi122 ceph-mon[29744]: pgmap v825: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:32.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:32 smithi149 ceph-mon[33409]: pgmap v825: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:32.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:32 smithi192 ceph-mon[33753]: pgmap v825: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:34.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:34 smithi122 ceph-mon[29744]: pgmap v826: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:34.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:34 smithi149 ceph-mon[33409]: pgmap v826: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:34.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:34 smithi192 ceph-mon[33753]: pgmap v826: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:36.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:36 smithi122 ceph-mon[29744]: pgmap v827: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:36.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:36 smithi149 ceph-mon[33409]: pgmap v827: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:36.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:36 smithi192 ceph-mon[33753]: pgmap v827: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:38.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:38 smithi122 ceph-mon[29744]: pgmap v828: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:38.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:38 smithi149 ceph-mon[33409]: pgmap v828: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:38.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:38 smithi192 ceph-mon[33753]: pgmap v828: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:40.671 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:40 smithi192 ceph-mon[33753]: pgmap v829: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:40.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:40 smithi122 ceph-mon[29744]: pgmap v829: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:40.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:40 smithi149 ceph-mon[33409]: pgmap v829: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:42.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:42 smithi122 ceph-mon[29744]: pgmap v830: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:42.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:42 smithi149 ceph-mon[33409]: pgmap v830: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:42.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:42 smithi192 ceph-mon[33753]: pgmap v830: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:44.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:44 smithi122 ceph-mon[29744]: pgmap v831: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:44.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:44 smithi149 ceph-mon[33409]: pgmap v831: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:44.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:44 smithi192 ceph-mon[33753]: pgmap v831: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:46.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:46 smithi122 ceph-mon[29744]: pgmap v832: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:46.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:46 smithi149 ceph-mon[33409]: pgmap v832: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:46.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:46 smithi192 ceph-mon[33753]: pgmap v832: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:48.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:48 smithi122 ceph-mon[29744]: pgmap v833: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:48.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:48 smithi149 ceph-mon[33409]: pgmap v833: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:48.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:48 smithi192 ceph-mon[33753]: pgmap v833: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:50.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:50 smithi122 ceph-mon[29744]: pgmap v834: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:50.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:50 smithi149 ceph-mon[33409]: pgmap v834: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:50.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:50 smithi192 ceph-mon[33753]: pgmap v834: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:10:52.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:52 smithi122 ceph-mon[29744]: pgmap v835: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:52.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:52 smithi149 ceph-mon[33409]: pgmap v835: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:52.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:52 smithi192 ceph-mon[33753]: pgmap v835: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:10:54.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:54 smithi122 ceph-mon[29744]: pgmap v836: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:54.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:54 smithi149 ceph-mon[33409]: pgmap v836: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:54.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:54 smithi192 ceph-mon[33753]: pgmap v836: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:10:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:56.661+0000 7fbf3e59c640 1 -- 172.21.15.122:0/2495789089 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf2c0b2920 msgr2=0x7fbf2c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).tick idle (900020417) for more than 900000000 us, fault. 2024-05-03T18:10:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:56.661+0000 7fbf3e59c640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf2c0b2920 0x7fbf2c0b4de0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbf40102960 tx=0x7fbf30008040 comp rx=0 tx=0).stop 2024-05-03T18:10:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:56.661+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf2c0b2920 msgr2=0x7fbf2c0b4de0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-03T18:10:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:56.661+0000 7fbf377fe640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf2c0b2920 0x7fbf2c0b4de0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:10:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:56.661+0000 7fbf377fe640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf2c0bc830 0x7fbf2c0bec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:10:56.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:56.661+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fbf38007e80 con 0x7fbf2c0bc830 2024-05-03T18:10:56.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:56.661+0000 7fbf3e59c640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf2c0bc830 0x7fbf2c0bec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:10:56.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:56.661+0000 7fbf3e59c640 1 --2- 172.21.15.122:0/2495789089 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fbf2c0bc830 0x7fbf2c0bec20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009920 tx=0x7fbf30012000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:10:56.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:56 smithi122 ceph-mon[29744]: pgmap v837: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:56.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:56.759+0000 7fbf367fc640 1 -- 172.21.15.122:0/2495789089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fbf240063d0 con 0x7fbf401034c0 2024-05-03T18:10:56.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:56.759+0000 7fbf377fe640 1 -- 172.21.15.122:0/2495789089 <== mon.0 v2:172.21.15.122:3300/0 12 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fbf240063d0 con 0x7fbf401034c0 2024-05-03T18:10:56.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:56 smithi149 ceph-mon[33409]: pgmap v837: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:56.853 INFO:teuthology.orchestra.run.smithi122.stderr:Error ETIMEDOUT: Connection timed out. Please check the client's permission and connection. 2024-05-03T18:10:56.860 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-05-03T18:10:56.860 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-05-03T18:10:57.085 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=fb7b858d864d4e5cb209641204f79fe7 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-05-03T18:10:57.087 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-03T18:10:57.097 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-03T18:10:57.097 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T18:10:57.097 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T18:10:57.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:56 smithi192 ceph-mon[33753]: pgmap v837: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-03T18:10:57.120 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-05-03T18:10:57.120 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T18:10:57.181 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-03T18:10:57.478 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:10:57.599 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:57 smithi122 ceph-mon[29744]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:10:57.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:57 smithi149 ceph-mon[33409]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:10:57.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:57 smithi192 ceph-mon[33753]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:10:58.703 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:10:58 smithi122 ceph-mon[29744]: pgmap v838: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:58.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:10:58 smithi149 ceph-mon[33409]: pgmap v838: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:58.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:10:58 smithi192 ceph-mon[33753]: pgmap v838: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:10:59.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.264+0000 7fb0db9e0700 1 -- 172.21.15.122:0/3354683957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4104100 msgr2=0x7fb0d4108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:10:59.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.264+0000 7fb0db9e0700 1 --2- 172.21.15.122:0/3354683957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4104100 0x7fb0d4108120 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40099c0 tx=0x7fb0c4009cd0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.264+0000 7fb0db9e0700 1 -- 172.21.15.122:0/3354683957 shutdown_connections 2024-05-03T18:10:59.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.264+0000 7fb0db9e0700 1 --2- 172.21.15.122:0/3354683957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0d4108850 0x7fb0d41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.264+0000 7fb0db9e0700 1 --2- 172.21.15.122:0/3354683957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0d4103750 0x7fb0d4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.264+0000 7fb0db9e0700 1 --2- 172.21.15.122:0/3354683957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4104100 0x7fb0d4108120 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.264+0000 7fb0db9e0700 1 -- 172.21.15.122:0/3354683957 >> 172.21.15.122:0/3354683957 conn(0x7fb0d407a600 msgr2=0x7fb0d407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:10:59.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.264+0000 7fb0db9e0700 1 -- 172.21.15.122:0/3354683957 shutdown_connections 2024-05-03T18:10:59.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0db9e0700 1 -- 172.21.15.122:0/3354683957 wait complete. 2024-05-03T18:10:59.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0db9e0700 1 Processor -- start 2024-05-03T18:10:59.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0db9e0700 1 -- start start 2024-05-03T18:10:59.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0db9e0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0d4103750 0x7fb0d40757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:10:59.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0db9e0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0d4104100 0x7fb0d4075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:10:59.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0db9e0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4108850 0x7fb0d4076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:10:59.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0db9e0700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4113630 con 0x7fb0d4108850 2024-05-03T18:10:59.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0db9e0700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41134b0 con 0x7fb0d4103750 2024-05-03T18:10:59.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0db9e0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41137b0 con 0x7fb0d4104100 2024-05-03T18:10:59.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0d9f7d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4108850 0x7fb0d4076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:10:59.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0d9f7d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4108850 0x7fb0d4076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56828/0 (socket says 172.21.15.122:56828) 2024-05-03T18:10:59.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0d9f7d700 1 -- 172.21.15.122:0/1460897380 learned_addr learned my addr 172.21.15.122:0/1460897380 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T18:10:59.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.265+0000 7fb0d977c700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0d4103750 0x7fb0d40757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:10:59.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0d8f7b700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0d4104100 0x7fb0d4075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:10:59.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0d9f7d700 1 -- 172.21.15.122:0/1460897380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0d4104100 msgr2=0x7fb0d4075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:10:59.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0d9f7d700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0d4104100 0x7fb0d4075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0d9f7d700 1 -- 172.21.15.122:0/1460897380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0d4103750 msgr2=0x7fb0d40757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:10:59.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0d9f7d700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0d4103750 0x7fb0d40757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0d9f7d700 1 -- 172.21.15.122:0/1460897380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d41a8030 con 0x7fb0d4108850 2024-05-03T18:10:59.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0d9f7d700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4108850 0x7fb0d4076220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc007ee0 tx=0x7fb0cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:10:59.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0ca7fc700 1 -- 172.21.15.122:0/1460897380 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc004bb0 con 0x7fb0d4108850 2024-05-03T18:10:59.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0ca7fc700 1 -- 172.21.15.122:0/1460897380 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0cc004d50 con 0x7fb0d4108850 2024-05-03T18:10:59.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0ca7fc700 1 -- 172.21.15.122:0/1460897380 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc005920 con 0x7fb0d4108850 2024-05-03T18:10:59.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0db9e0700 1 -- 172.21.15.122:0/1460897380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d41a8320 con 0x7fb0d4108850 2024-05-03T18:10:59.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.266+0000 7fb0db9e0700 1 -- 172.21.15.122:0/1460897380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d41a87e0 con 0x7fb0d4108850 2024-05-03T18:10:59.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.267+0000 7fb0db9e0700 1 -- 172.21.15.122:0/1460897380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d410ba20 con 0x7fb0d4108850 2024-05-03T18:10:59.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.268+0000 7fb0ca7fc700 1 -- 172.21.15.122:0/1460897380 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb0cc01a020 con 0x7fb0d4108850 2024-05-03T18:10:59.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.269+0000 7fb0ca7fc700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb0c00b2910 0x7fb0c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:10:59.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.269+0000 7fb0d977c700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb0c00b2910 0x7fb0c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:10:59.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.269+0000 7fb0ca7fc700 1 -- 172.21.15.122:0/1460897380 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fb0cc0e1f30 con 0x7fb0d4108850 2024-05-03T18:10:59.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.270+0000 7fb0d977c700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb0c00b2910 0x7fb0c00b4dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0004630 tx=0x7fb0d000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:10:59.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.273+0000 7fb0ca7fc700 1 -- 172.21.15.122:0/1460897380 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb0cc010070 con 0x7fb0d4108850 2024-05-03T18:10:59.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.499+0000 7fb0db9e0700 1 -- 172.21.15.122:0/1460897380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb0d410b400 con 0x7fb0d4108850 2024-05-03T18:10:59.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.499+0000 7fb0ca7fc700 1 -- 172.21.15.122:0/1460897380 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v79) ==== 81+0+594 (secure 0 0 0) 0x7fb0cc0a9bf0 con 0x7fb0d4108850 2024-05-03T18:10:59.498 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/2089907395 2024-05-04T17:45:18.454406+0000 2024-05-03T18:10:59.498 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6801/1659741627 2024-05-04T17:45:18.454406+0000 2024-05-03T18:10:59.499 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/2001860625 2024-05-04T17:44:52.244049+0000 2024-05-03T18:10:59.499 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/1888284988 2024-05-04T17:44:52.244049+0000 2024-05-03T18:10:59.499 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/256574472 2024-05-04T17:44:52.244049+0000 2024-05-03T18:10:59.499 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6801/80220221 2024-05-04T17:44:52.244049+0000 2024-05-03T18:10:59.499 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/64546794 2024-05-04T17:45:18.454406+0000 2024-05-03T18:10:59.499 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6800/1659741627 2024-05-04T17:45:18.454406+0000 2024-05-03T18:10:59.499 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6800/80220221 2024-05-04T17:44:52.244049+0000 2024-05-03T18:10:59.499 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/584846864 2024-05-04T17:45:18.454406+0000 2024-05-03T18:10:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 -- 172.21.15.122:0/1460897380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb0c00b2910 msgr2=0x7fb0c00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:10:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb0c00b2910 0x7fb0c00b4dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0004630 tx=0x7fb0d000b3c0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 -- 172.21.15.122:0/1460897380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4108850 msgr2=0x7fb0d4076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:10:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4108850 0x7fb0d4076220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc007ee0 tx=0x7fb0cc00d3b0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 -- 172.21.15.122:0/1460897380 shutdown_connections 2024-05-03T18:10:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fb0c00b2910 0x7fb0c00b4dd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4108850 0x7fb0d4076220 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0d4103750 0x7fb0d40757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 --2- 172.21.15.122:0/1460897380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0d4104100 0x7fb0d4075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:10:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 -- 172.21.15.122:0/1460897380 >> 172.21.15.122:0/1460897380 conn(0x7fb0d407a600 msgr2=0x7fb0d41099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:10:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 -- 172.21.15.122:0/1460897380 shutdown_connections 2024-05-03T18:10:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:10:59.502+0000 7fb0db9e0700 1 -- 172.21.15.122:0/1460897380 wait complete. 2024-05-03T18:10:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:listed 10 entries 2024-05-03T18:11:00.174 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-03T18:11:00.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:11:00.174 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-03T18:11:00.206 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-05-03T18:11:00.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:11:00.206 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-03T18:11:00.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:11:00.271 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-05-03T18:11:00.327 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-03T18:11:00.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:11:00.327 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-03T18:11:00.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:11:00.391 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-05-03T18:11:00.493 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-03T18:11:00.504 INFO:tasks.cephadm:Teardown begin 2024-05-03T18:11:00.504 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-03T18:11:00.504 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:11:00.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:00 smithi122 ceph-mon[29744]: pgmap v839: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:11:00.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:00 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1460897380' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:11:00.745 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:11:00.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:00 smithi149 ceph-mon[33409]: pgmap v839: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:11:00.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:00 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/1460897380' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:11:00.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:00 smithi192 ceph-mon[33753]: pgmap v839: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-03T18:11:00.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:00 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/1460897380' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:11:02.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.355+0000 7f9b7baab700 1 -- 172.21.15.122:0/411559702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b74109510 msgr2=0x7f9b741098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:02.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.355+0000 7f9b7baab700 1 --2- 172.21.15.122:0/411559702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b74109510 0x7f9b741098f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009990 tx=0x7f9b70009ca0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.355+0000 7f9b7baab700 1 -- 172.21.15.122:0/411559702 shutdown_connections 2024-05-03T18:11:02.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.355+0000 7f9b7baab700 1 --2- 172.21.15.122:0/411559702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b7406bb40 0x7f9b74110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.355+0000 7f9b7baab700 1 --2- 172.21.15.122:0/411559702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b74109510 0x7f9b741098f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.355+0000 7f9b7baab700 1 --2- 172.21.15.122:0/411559702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b7406b180 0x7f9b7406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.355+0000 7f9b7baab700 1 -- 172.21.15.122:0/411559702 >> 172.21.15.122:0/411559702 conn(0x7f9b74079350 msgr2=0x7f9b74079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:02.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.355+0000 7f9b7baab700 1 -- 172.21.15.122:0/411559702 shutdown_connections 2024-05-03T18:11:02.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.355+0000 7f9b7baab700 1 -- 172.21.15.122:0/411559702 wait complete. 2024-05-03T18:11:02.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b7baab700 1 Processor -- start 2024-05-03T18:11:02.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b7baab700 1 -- start start 2024-05-03T18:11:02.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b7baab700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b7406b180 0x7f9b7419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:02.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b7baab700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b7406bb40 0x7f9b7419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:02.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b7baab700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b74109510 0x7f9b74199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:02.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b7baab700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b74113720 con 0x7f9b7406b180 2024-05-03T18:11:02.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b7baab700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9b741135a0 con 0x7f9b7406bb40 2024-05-03T18:11:02.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b7baab700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9b741138a0 con 0x7f9b74109510 2024-05-03T18:11:02.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b79847700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b7406b180 0x7f9b7419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:02.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b79847700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b7406b180 0x7f9b7419f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56850/0 (socket says 172.21.15.122:56850) 2024-05-03T18:11:02.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b79847700 1 -- 172.21.15.122:0/3647651158 learned_addr learned my addr 172.21.15.122:0/3647651158 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T18:11:02.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b79046700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b7406bb40 0x7f9b7419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:02.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b7a048700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b74109510 0x7f9b74199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:02.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b79847700 1 -- 172.21.15.122:0/3647651158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b74109510 msgr2=0x7f9b74199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:02.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.356+0000 7f9b79847700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b74109510 0x7f9b74199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.357+0000 7f9b79847700 1 -- 172.21.15.122:0/3647651158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b7406bb40 msgr2=0x7f9b7419fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:02.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.357+0000 7f9b79847700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b7406bb40 0x7f9b7419fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.357+0000 7f9b79847700 1 -- 172.21.15.122:0/3647651158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7419a240 con 0x7f9b7406b180 2024-05-03T18:11:02.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.357+0000 7f9b7a048700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b74109510 0x7f9b74199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:11:02.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.357+0000 7f9b79847700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b7406b180 0x7f9b7419f920 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000b5c0 tx=0x7f9b70004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:02.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.357+0000 7f9b6affd700 1 -- 172.21.15.122:0/3647651158 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70026070 con 0x7f9b7406b180 2024-05-03T18:11:02.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.357+0000 7f9b7baab700 1 -- 172.21.15.122:0/3647651158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7419a4d0 con 0x7f9b7406b180 2024-05-03T18:11:02.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.357+0000 7f9b6affd700 1 -- 172.21.15.122:0/3647651158 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b70004170 con 0x7f9b7406b180 2024-05-03T18:11:02.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.357+0000 7f9b6affd700 1 -- 172.21.15.122:0/3647651158 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70005250 con 0x7f9b7406b180 2024-05-03T18:11:02.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.357+0000 7f9b7baab700 1 -- 172.21.15.122:0/3647651158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b741a67f0 con 0x7f9b7406b180 2024-05-03T18:11:02.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.358+0000 7f9b7baab700 1 -- 172.21.15.122:0/3647651158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7410b9f0 con 0x7f9b7406b180 2024-05-03T18:11:02.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.359+0000 7f9b6affd700 1 -- 172.21.15.122:0/3647651158 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f9b70029070 con 0x7f9b7406b180 2024-05-03T18:11:02.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.360+0000 7f9b6affd700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9b600b2910 0x7f9b600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:02.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.360+0000 7f9b79046700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9b600b2910 0x7f9b600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:02.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.360+0000 7f9b6affd700 1 -- 172.21.15.122:0/3647651158 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f9b700e2f30 con 0x7f9b7406b180 2024-05-03T18:11:02.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.360+0000 7f9b79046700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9b600b2910 0x7f9b600b4dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b64006fd0 tx=0x7f9b64008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:02.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.363+0000 7f9b6affd700 1 -- 172.21.15.122:0/3647651158 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9b70013070 con 0x7f9b7406b180 2024-05-03T18:11:02.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.618+0000 7f9b7baab700 1 -- 172.21.15.122:0/3647651158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b7406a190 con 0x7f9b7406b180 2024-05-03T18:11:02.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.619+0000 7f9b6affd700 1 -- 172.21.15.122:0/3647651158 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11892 (secure 0 0 0) 0x7f9b700aac80 con 0x7f9b7406b180 2024-05-03T18:11:02.618 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T18:11:02.618 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/555236113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":555236113},{"type":"v1","addr":"172.21.15.122:6839","nonce":555236113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1783292878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1783292878},{"type":"v1","addr":"172.21.15.122:6841","nonce":1783292878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T17:52:37.616728+0000","modified":"2024-05-03T17:55:56.222099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24463,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24410":{"gid":24410,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1529632960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1529632960},{"type":"v1","addr":"172.21.15.149:6839","nonce":1529632960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1429818440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1429818440},{"type":"v1","addr":"172.21.15.192:6839","nonce":1429818440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24463,24481,24457]},"id":1}]} 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 -- 172.21.15.122:0/3647651158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9b600b2910 msgr2=0x7f9b600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9b600b2910 0x7f9b600b4dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b64006fd0 tx=0x7f9b64008040 comp rx=0 tx=0).stop 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 -- 172.21.15.122:0/3647651158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b7406b180 msgr2=0x7f9b7419f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b7406b180 0x7f9b7419f920 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000b5c0 tx=0x7f9b70004630 comp rx=0 tx=0).stop 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 -- 172.21.15.122:0/3647651158 shutdown_connections 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f9b600b2910 0x7f9b600b4dd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b74109510 0x7f9b74199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b7406b180 0x7f9b7419f920 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 --2- 172.21.15.122:0/3647651158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b7406bb40 0x7f9b7419fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 -- 172.21.15.122:0/3647651158 >> 172.21.15.122:0/3647651158 conn(0x7f9b74079350 msgr2=0x7f9b74101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 -- 172.21.15.122:0/3647651158 shutdown_connections 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:02.621+0000 7f9b7baab700 1 -- 172.21.15.122:0/3647651158 wait complete. 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 25 2024-05-03T18:11:02.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:02 smithi122 ceph-mon[29744]: pgmap v840: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:11:02.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:02 smithi149 ceph-mon[33409]: pgmap v840: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:11:02.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:02 smithi192 ceph-mon[33753]: pgmap v840: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T18:11:03.282 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:11:03.519 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:11:03.641 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:03 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3647651158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:11:03.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:03 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3647651158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:11:03.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:03 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3647651158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:11:04.591 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:04 smithi122 ceph-mon[29744]: pgmap v841: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-03T18:11:04.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:04 smithi149 ceph-mon[33409]: pgmap v841: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-03T18:11:04.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:04 smithi192 ceph-mon[33753]: pgmap v841: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-03T18:11:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.174+0000 7f652a155700 1 -- 172.21.15.122:0/2854897383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6524102450 msgr2=0x7f6524110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:05.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.174+0000 7f652a155700 1 --2- 172.21.15.122:0/2854897383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6524102450 0x7f6524110340 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f651c00b0a0 tx=0x7f651c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.174+0000 7f652a155700 1 -- 172.21.15.122:0/2854897383 shutdown_connections 2024-05-03T18:11:05.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.175+0000 7f652a155700 1 --2- 172.21.15.122:0/2854897383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6524102450 0x7f6524110340 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.175+0000 7f652a155700 1 --2- 172.21.15.122:0/2854897383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6524109520 0x7f6524109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.175+0000 7f652a155700 1 --2- 172.21.15.122:0/2854897383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524101a90 0x7f6524101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.175+0000 7f652a155700 1 -- 172.21.15.122:0/2854897383 >> 172.21.15.122:0/2854897383 conn(0x7f6524078a10 msgr2=0x7f65240ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:05.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.175+0000 7f652a155700 1 -- 172.21.15.122:0/2854897383 shutdown_connections 2024-05-03T18:11:05.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.175+0000 7f652a155700 1 -- 172.21.15.122:0/2854897383 wait complete. 2024-05-03T18:11:05.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.175+0000 7f652a155700 1 Processor -- start 2024-05-03T18:11:05.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.175+0000 7f652a155700 1 -- start start 2024-05-03T18:11:05.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f652a155700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6524101a90 0x7f652419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:05.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f652a155700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102450 0x7f652419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:05.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f652a155700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6524109520 0x7f6524199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:05.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f652a155700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6524113650 con 0x7f6524101a90 2024-05-03T18:11:05.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f652a155700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65241134d0 con 0x7f6524102450 2024-05-03T18:11:05.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f652a155700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f65241137d0 con 0x7f6524109520 2024-05-03T18:11:05.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f65237fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6524101a90 0x7f652419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:05.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f65237fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6524101a90 0x7f652419f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56872/0 (socket says 172.21.15.122:56872) 2024-05-03T18:11:05.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f65237fe700 1 -- 172.21.15.122:0/723696980 learned_addr learned my addr 172.21.15.122:0/723696980 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T18:11:05.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f6522ffd700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102450 0x7f652419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:05.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f6523fff700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6524109520 0x7f6524199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:05.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f6522ffd700 1 -- 172.21.15.122:0/723696980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6524109520 msgr2=0x7f6524199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:05.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f6522ffd700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6524109520 0x7f6524199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f6522ffd700 1 -- 172.21.15.122:0/723696980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6524101a90 msgr2=0x7f652419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:05.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f6522ffd700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6524101a90 0x7f652419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f6522ffd700 1 -- 172.21.15.122:0/723696980 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652419a0b0 con 0x7f6524102450 2024-05-03T18:11:05.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f6523fff700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6524109520 0x7f6524199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:11:05.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f65237fe700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6524101a90 0x7f652419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:11:05.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.176+0000 7f6522ffd700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102450 0x7f652419fc80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f651800e9f0 tx=0x7f651800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:05.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.177+0000 7f6520ff9700 1 -- 172.21.15.122:0/723696980 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651800cb70 con 0x7f6524102450 2024-05-03T18:11:05.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.177+0000 7f6520ff9700 1 -- 172.21.15.122:0/723696980 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65180044d0 con 0x7f6524102450 2024-05-03T18:11:05.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.177+0000 7f652a155700 1 -- 172.21.15.122:0/723696980 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652419a3a0 con 0x7f6524102450 2024-05-03T18:11:05.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.177+0000 7f6520ff9700 1 -- 172.21.15.122:0/723696980 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518003e00 con 0x7f6524102450 2024-05-03T18:11:05.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.177+0000 7f652a155700 1 -- 172.21.15.122:0/723696980 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65241a6610 con 0x7f6524102450 2024-05-03T18:11:05.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.178+0000 7f652a155700 1 -- 172.21.15.122:0/723696980 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f652406a190 con 0x7f6524102450 2024-05-03T18:11:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.179+0000 7f6520ff9700 1 -- 172.21.15.122:0/723696980 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f651800cd10 con 0x7f6524102450 2024-05-03T18:11:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.180+0000 7f6520ff9700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f650c0b2990 0x7f650c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.180+0000 7f65237fe700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f650c0b2990 0x7f650c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.180+0000 7f6520ff9700 1 -- 172.21.15.122:0/723696980 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7f65180e25b0 con 0x7f6524102450 2024-05-03T18:11:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.181+0000 7f65237fe700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f650c0b2990 0x7f650c0b4e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6524108600 tx=0x7f6514009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.184+0000 7f6520ff9700 1 -- 172.21.15.122:0/723696980 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f65180aa200 con 0x7f6524102450 2024-05-03T18:11:05.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.449+0000 7f652a155700 1 -- 172.21.15.122:0/723696980 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f652402e510 con 0x7f6524102450 2024-05-03T18:11:05.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.450+0000 7f6520ff9700 1 -- 172.21.15.122:0/723696980 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11892 (secure 0 0 0) 0x7f65180a9b00 con 0x7f6524102450 2024-05-03T18:11:05.450 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T18:11:05.450 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2168395467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2168395467},{"type":"v1","addr":"172.21.15.122:6835","nonce":2168395467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3093239187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3093239187},{"type":"v1","addr":"172.21.15.122:6837","nonce":3093239187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/555236113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":555236113},{"type":"v1","addr":"172.21.15.122:6839","nonce":555236113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1783292878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1783292878},{"type":"v1","addr":"172.21.15.122:6841","nonce":1783292878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24371,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3370166837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3370166837},{"type":"v1","addr":"172.21.15.149:6833","nonce":3370166837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1470504763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1470504763},{"type":"v1","addr":"172.21.15.149:6835","nonce":1470504763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T17:52:37.616728+0000","modified":"2024-05-03T17:55:56.222099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24481,"mds_1":24463,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24410":{"gid":24410,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2599524712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2599524712},{"type":"v1","addr":"172.21.15.149:6837","nonce":2599524712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/899389349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":899389349},{"type":"v1","addr":"172.21.15.192:6833","nonce":899389349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1529632960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1529632960},{"type":"v1","addr":"172.21.15.149:6839","nonce":1529632960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.192:6835/2756708496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2756708496},{"type":"v1","addr":"172.21.15.192:6835","nonce":2756708496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.192:6837/3476407415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3476407415},{"type":"v1","addr":"172.21.15.192:6837","nonce":3476407415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1429818440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1429818440},{"type":"v1","addr":"172.21.15.192:6839","nonce":1429818440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24463,24481,24457]},"id":1}]} 2024-05-03T18:11:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 -- 172.21.15.122:0/723696980 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f650c0b2990 msgr2=0x7f650c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f650c0b2990 0x7f650c0b4e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6524108600 tx=0x7f6514009260 comp rx=0 tx=0).stop 2024-05-03T18:11:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 -- 172.21.15.122:0/723696980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102450 msgr2=0x7f652419fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102450 0x7f652419fc80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f651800e9f0 tx=0x7f651800ed00 comp rx=0 tx=0).stop 2024-05-03T18:11:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 -- 172.21.15.122:0/723696980 shutdown_connections 2024-05-03T18:11:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f650c0b2990 0x7f650c0b4e50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6524109520 0x7f6524199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6524101a90 0x7f652419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 --2- 172.21.15.122:0/723696980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102450 0x7f652419fc80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 -- 172.21.15.122:0/723696980 >> 172.21.15.122:0/723696980 conn(0x7f6524078a10 msgr2=0x7f6524100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:05.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 -- 172.21.15.122:0/723696980 shutdown_connections 2024-05-03T18:11:05.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:05.453+0000 7f652a155700 1 -- 172.21.15.122:0/723696980 wait complete. 2024-05-03T18:11:05.454 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 25 2024-05-03T18:11:05.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:05 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:11:05.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:05 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:11:05.868 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:05 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:11:06.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:11:06.371 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:11:06.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:06 smithi122 ceph-mon[29744]: pgmap v842: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-03T18:11:06.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:06 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/723696980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:11:06.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.714 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.714 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:11:06.714 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:11:06.714 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:06 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:11:06.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:06 smithi149 ceph-mon[33409]: pgmap v842: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-03T18:11:06.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:06 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/723696980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:11:06.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:11:06.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:11:06.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:06 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:11:06.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:06 smithi192 ceph-mon[33753]: pgmap v842: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-03T18:11:06.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:06 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/723696980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:11:06.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:11:06.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:11:06.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:11:06.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:06 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' 2024-05-03T18:11:07.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:07 smithi149 ceph-mon[33409]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-05-03T18:11:07.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:07 smithi149 ceph-mon[33409]: Unable to set osd_memory_target on smithi122 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-03T18:11:07.815 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:07 smithi122 ceph-mon[29744]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-05-03T18:11:07.815 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:07 smithi122 ceph-mon[29744]: Unable to set osd_memory_target on smithi122 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-03T18:11:07.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:07 smithi192 ceph-mon[33753]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-05-03T18:11:07.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:07 smithi192 ceph-mon[33753]: Unable to set osd_memory_target on smithi122 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-03T18:11:08.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.034+0000 7ffa38cea700 1 -- 172.21.15.122:0/4264153669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa34102390 msgr2=0x7ffa34102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:08.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.034+0000 7ffa38cea700 1 --2- 172.21.15.122:0/4264153669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa34102390 0x7ffa34102810 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800b0a0 tx=0x7ffa2800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:11:08.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.035+0000 7ffa38cea700 1 -- 172.21.15.122:0/4264153669 shutdown_connections 2024-05-03T18:11:08.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.035+0000 7ffa38cea700 1 --2- 172.21.15.122:0/4264153669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa34102d50 0x7ffa3410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:08.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.035+0000 7ffa38cea700 1 --2- 172.21.15.122:0/4264153669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa34108400 0x7ffa341087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:08.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.035+0000 7ffa38cea700 1 --2- 172.21.15.122:0/4264153669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa34102390 0x7ffa34102810 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.035+0000 7ffa38cea700 1 -- 172.21.15.122:0/4264153669 >> 172.21.15.122:0/4264153669 conn(0x7ffa34079310 msgr2=0x7ffa34079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.035+0000 7ffa38cea700 1 -- 172.21.15.122:0/4264153669 shutdown_connections 2024-05-03T18:11:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.035+0000 7ffa38cea700 1 -- 172.21.15.122:0/4264153669 wait complete. 2024-05-03T18:11:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.035+0000 7ffa38cea700 1 Processor -- start 2024-05-03T18:11:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.035+0000 7ffa38cea700 1 -- start start 2024-05-03T18:11:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa38cea700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa34102390 0x7ffa3419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa38cea700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa34102d50 0x7ffa3419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa38cea700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa34108400 0x7ffa341999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:08.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa38cea700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34113640 con 0x7ffa34102390 2024-05-03T18:11:08.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa38cea700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa341134c0 con 0x7ffa34108400 2024-05-03T18:11:08.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa38cea700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa341137c0 con 0x7ffa34102d50 2024-05-03T18:11:08.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa3259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa34102390 0x7ffa3419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:08.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa3259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa34102390 0x7ffa3419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56900/0 (socket says 172.21.15.122:56900) 2024-05-03T18:11:08.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa3259c700 1 -- 172.21.15.122:0/1855840165 learned_addr learned my addr 172.21.15.122:0/1855840165 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T18:11:08.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa31d9b700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa34102d50 0x7ffa3419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:08.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa32d9d700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa34108400 0x7ffa341999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:08.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa3259c700 1 -- 172.21.15.122:0/1855840165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa34102d50 msgr2=0x7ffa3419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:08.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa3259c700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa34102d50 0x7ffa3419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:08.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa3259c700 1 -- 172.21.15.122:0/1855840165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa34108400 msgr2=0x7ffa341999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:08.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa3259c700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa34108400 0x7ffa341999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:08.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa3259c700 1 -- 172.21.15.122:0/1855840165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3419a100 con 0x7ffa34102390 2024-05-03T18:11:08.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa3259c700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa34102390 0x7ffa3419f7e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00d950 tx=0x7ffa2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:08.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa237fe700 1 -- 172.21.15.122:0/1855840165 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c016070 con 0x7ffa34102390 2024-05-03T18:11:08.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa237fe700 1 -- 172.21.15.122:0/1855840165 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2c009df0 con 0x7ffa34102390 2024-05-03T18:11:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa38cea700 1 -- 172.21.15.122:0/1855840165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3419a3f0 con 0x7ffa34102390 2024-05-03T18:11:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa237fe700 1 -- 172.21.15.122:0/1855840165 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00f3c0 con 0x7ffa34102390 2024-05-03T18:11:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.036+0000 7ffa38cea700 1 -- 172.21.15.122:0/1855840165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa341a66b0 con 0x7ffa34102390 2024-05-03T18:11:08.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.037+0000 7ffa38cea700 1 -- 172.21.15.122:0/1855840165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3406a190 con 0x7ffa34102390 2024-05-03T18:11:08.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.039+0000 7ffa237fe700 1 -- 172.21.15.122:0/1855840165 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7ffa2c019070 con 0x7ffa34102390 2024-05-03T18:11:08.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.039+0000 7ffa237fe700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffa1c0b2910 0x7ffa1c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:08.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.039+0000 7ffa31d9b700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffa1c0b2910 0x7ffa1c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:08.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.040+0000 7ffa237fe700 1 -- 172.21.15.122:0/1855840165 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7ffa2c0e1df0 con 0x7ffa34102390 2024-05-03T18:11:08.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.040+0000 7ffa31d9b700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffa1c0b2910 0x7ffa1c0b4dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009e60 tx=0x7ffa2800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:08.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.043+0000 7ffa237fe700 1 -- 172.21.15.122:0/1855840165 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ffa2c0a9ab0 con 0x7ffa34102390 2024-05-03T18:11:08.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.253+0000 7ffa38cea700 1 -- 172.21.15.122:0/1855840165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa3402e510 con 0x7ffa34102390 2024-05-03T18:11:08.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.255+0000 7ffa237fe700 1 -- 172.21.15.122:0/1855840165 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22622 (secure 0 0 0) 0x7ffa2c0a93b0 con 0x7ffa34102390 2024-05-03T18:11:08.254 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-05-03T18:11:08.254 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":79,"fsid":"a46fc846-0974-11ef-bc97-c7b262605968","created":"2024-05-03T17:44:17.415571+0000","modified":"2024-05-03T17:53:19.863832+0000","last_up_change":"2024-05-03T17:51:28.839117+0000","last_in_change":"2024-05-03T17:51:07.954070+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T17:47:40.539967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T17:52:31.646919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T17:52:34.503067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4b08628-0a29-44aa-b8b9-ed9657f12be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6803","nonce":1329364610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6805","nonce":1329364610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6809","nonce":1329364610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1329364610},{"type":"v1","addr":"172.21.15.122:6807","nonce":1329364610}]},"public_addr":"172.21.15.122:6803/1329364610","cluster_addr":"172.21.15.122:6805/1329364610","heartbeat_back_addr":"172.21.15.122:6809/1329364610","heartbeat_front_addr":"172.21.15.122:6807/1329364610","state":["exists","up"]},{"osd":1,"uuid":"ba550787-ce76-4944-8ebd-97e19590bb3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6801","nonce":1402463302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6803","nonce":1402463302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6807","nonce":1402463302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1402463302},{"type":"v1","addr":"172.21.15.149:6805","nonce":1402463302}]},"public_addr":"172.21.15.149:6801/1402463302","cluster_addr":"172.21.15.149:6803/1402463302","heartbeat_back_addr":"172.21.15.149:6807/1402463302","heartbeat_front_addr":"172.21.15.149:6805/1402463302","state":["exists","up"]},{"osd":2,"uuid":"35ae7c47-d710-47c5-b0ec-91fa12b5be07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6801","nonce":3757191110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6803","nonce":3757191110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6807","nonce":3757191110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3757191110},{"type":"v1","addr":"172.21.15.192:6805","nonce":3757191110}]},"public_addr":"172.21.15.192:6801/3757191110","cluster_addr":"172.21.15.192:6803/3757191110","heartbeat_back_addr":"172.21.15.192:6807/3757191110","heartbeat_front_addr":"172.21.15.192:6805/3757191110","state":["exists","up"]},{"osd":3,"uuid":"9973c0c1-ae73-497a-845e-664a20666df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6811","nonce":2359143764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6813","nonce":2359143764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6817","nonce":2359143764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2359143764},{"type":"v1","addr":"172.21.15.122:6815","nonce":2359143764}]},"public_addr":"172.21.15.122:6811/2359143764","cluster_addr":"172.21.15.122:6813/2359143764","heartbeat_back_addr":"172.21.15.122:6817/2359143764","heartbeat_front_addr":"172.21.15.122:6815/2359143764","state":["exists","up"]},{"osd":4,"uuid":"da71dc18-94b9-4f7d-a061-7cc7b77db282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3627037532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3627037532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3627037532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3627037532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3627037532}]},"public_addr":"172.21.15.149:6809/3627037532","cluster_addr":"172.21.15.149:6811/3627037532","heartbeat_back_addr":"172.21.15.149:6815/3627037532","heartbeat_front_addr":"172.21.15.149:6813/3627037532","state":["exists","up"]},{"osd":5,"uuid":"bcdd6808-36e2-46f2-b6d0-5a6bd90fe0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6809","nonce":455779485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6811","nonce":455779485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6815","nonce":455779485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":455779485},{"type":"v1","addr":"172.21.15.192:6813","nonce":455779485}]},"public_addr":"172.21.15.192:6809/455779485","cluster_addr":"172.21.15.192:6811/455779485","heartbeat_back_addr":"172.21.15.192:6815/455779485","heartbeat_front_addr":"172.21.15.192:6813/455779485","state":["exists","up"]},{"osd":6,"uuid":"4150bfe5-a0e4-4bae-914a-75598f017519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6819","nonce":3419421998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6821","nonce":3419421998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6825","nonce":3419421998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3419421998},{"type":"v1","addr":"172.21.15.122:6823","nonce":3419421998}]},"public_addr":"172.21.15.122:6819/3419421998","cluster_addr":"172.21.15.122:6821/3419421998","heartbeat_back_addr":"172.21.15.122:6825/3419421998","heartbeat_front_addr":"172.21.15.122:6823/3419421998","state":["exists","up"]},{"osd":7,"uuid":"71e01812-e511-44a0-a143-2afca510ae8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6817","nonce":139747549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6819","nonce":139747549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6823","nonce":139747549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":139747549},{"type":"v1","addr":"172.21.15.149:6821","nonce":139747549}]},"public_addr":"172.21.15.149:6817/139747549","cluster_addr":"172.21.15.149:6819/139747549","heartbeat_back_addr":"172.21.15.149:6823/139747549","heartbeat_front_addr":"172.21.15.149:6821/139747549","state":["exists","up"]},{"osd":8,"uuid":"5294516d-fac6-41d0-a635-6bc1d7eca585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6817","nonce":2080816744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6819","nonce":2080816744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6823","nonce":2080816744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2080816744},{"type":"v1","addr":"172.21.15.192:6821","nonce":2080816744}]},"public_addr":"172.21.15.192:6817/2080816744","cluster_addr":"172.21.15.192:6819/2080816744","heartbeat_back_addr":"172.21.15.192:6823/2080816744","heartbeat_front_addr":"172.21.15.192:6821/2080816744","state":["exists","up"]},{"osd":9,"uuid":"f5d5fcfd-b9e1-4e8f-b5ce-73b7fd889126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6827","nonce":1773691421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6829","nonce":1773691421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6833","nonce":1773691421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1773691421},{"type":"v1","addr":"172.21.15.122:6831","nonce":1773691421}]},"public_addr":"172.21.15.122:6827/1773691421","cluster_addr":"172.21.15.122:6829/1773691421","heartbeat_back_addr":"172.21.15.122:6833/1773691421","heartbeat_front_addr":"172.21.15.122:6831/1773691421","state":["exists","up"]},{"osd":10,"uuid":"af88e50f-98f1-460d-856b-a920dd2ae514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6825","nonce":1954186137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6827","nonce":1954186137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6831","nonce":1954186137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1954186137},{"type":"v1","addr":"172.21.15.149:6829","nonce":1954186137}]},"public_addr":"172.21.15.149:6825/1954186137","cluster_addr":"172.21.15.149:6827/1954186137","heartbeat_back_addr":"172.21.15.149:6831/1954186137","heartbeat_front_addr":"172.21.15.149:6829/1954186137","state":["exists","up"]},{"osd":11,"uuid":"b54c1b4d-cdb5-42d9-979a-47f9a870b5cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6825","nonce":2098474209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6827","nonce":2098474209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6831","nonce":2098474209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2098474209},{"type":"v1","addr":"172.21.15.192:6829","nonce":2098474209}]},"public_addr":"172.21.15.192:6825/2098474209","cluster_addr":"172.21.15.192:6827/2098474209","heartbeat_back_addr":"172.21.15.192:6831/2098474209","heartbeat_front_addr":"172.21.15.192:6829/2098474209","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:46:50.049769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:13.518485+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:47:37.473023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:01.809640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:26.648514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:48:51.268056+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:16.294834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:49:42.079065+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:07.609271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:33.885127+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:50:59.982708+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T17:51:26.461369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/2089907395":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6801/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:0/2001860625":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/1888284988":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/256574472":"2024-05-04T17:44:52.244049+0000","172.21.15.122:6801/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/64546794":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/1659741627":"2024-05-04T17:45:18.454406+0000","172.21.15.122:6800/80220221":"2024-05-04T17:44:52.244049+0000","172.21.15.122:0/584846864":"2024-05-04T17:45:18.454406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:11:08.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.257+0000 7ffa38cea700 1 -- 172.21.15.122:0/1855840165 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffa1c0b2910 msgr2=0x7ffa1c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:08.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.257+0000 7ffa38cea700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffa1c0b2910 0x7ffa1c0b4dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009e60 tx=0x7ffa2800b7b0 comp rx=0 tx=0).stop 2024-05-03T18:11:08.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.257+0000 7ffa38cea700 1 -- 172.21.15.122:0/1855840165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa34102390 msgr2=0x7ffa3419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:08.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.257+0000 7ffa38cea700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa34102390 0x7ffa3419f7e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00d950 tx=0x7ffa2c00dc60 comp rx=0 tx=0).stop 2024-05-03T18:11:08.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.257+0000 7ffa38cea700 1 -- 172.21.15.122:0/1855840165 shutdown_connections 2024-05-03T18:11:08.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.257+0000 7ffa38cea700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7ffa1c0b2910 0x7ffa1c0b4dd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:08.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.257+0000 7ffa38cea700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa34108400 0x7ffa341999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:08.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.257+0000 7ffa38cea700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa34102390 0x7ffa3419f7e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:08.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.257+0000 7ffa38cea700 1 --2- 172.21.15.122:0/1855840165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa34102d50 0x7ffa3419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:08.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.257+0000 7ffa38cea700 1 -- 172.21.15.122:0/1855840165 >> 172.21.15.122:0/1855840165 conn(0x7ffa34079310 msgr2=0x7ffa34101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:08.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.257+0000 7ffa38cea700 1 -- 172.21.15.122:0/1855840165 shutdown_connections 2024-05-03T18:11:08.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:08.258+0000 7ffa38cea700 1 -- 172.21.15.122:0/1855840165 wait complete. 2024-05-03T18:11:08.530 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:08 smithi122 ceph-mon[29744]: pgmap v843: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-03T18:11:08.530 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:08 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1855840165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:11:08.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:08 smithi149 ceph-mon[33409]: pgmap v843: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-03T18:11:08.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:08 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/1855840165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:11:08.834 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-03T18:11:08.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:08 smithi192 ceph-mon[33753]: pgmap v843: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-03T18:11:08.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:08 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/1855840165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:11:09.071 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:11:10.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.699+0000 7fafbdad4700 1 -- 172.21.15.122:0/963796855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb81076e0 msgr2=0x7fafb810f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:10.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.699+0000 7fafbdad4700 1 --2- 172.21.15.122:0/963796855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb81076e0 0x7fafb810f2b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fafb000b0d0 tx=0x7fafb000b3e0 comp rx=0 tx=0).stop 2024-05-03T18:11:10.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.699+0000 7fafbdad4700 1 -- 172.21.15.122:0/963796855 shutdown_connections 2024-05-03T18:11:10.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.699+0000 7fafbdad4700 1 --2- 172.21.15.122:0/963796855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb81076e0 0x7fafb810f2b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:10.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.699+0000 7fafbdad4700 1 --2- 172.21.15.122:0/963796855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb8102690 0x7fafb8102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:10.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.699+0000 7fafbdad4700 1 --2- 172.21.15.122:0/963796855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb8103040 0x7fafb8107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:10.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.699+0000 7fafbdad4700 1 -- 172.21.15.122:0/963796855 >> 172.21.15.122:0/963796855 conn(0x7fafb80783f0 msgr2=0x7fafb8078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:10.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.700+0000 7fafbdad4700 1 -- 172.21.15.122:0/963796855 shutdown_connections 2024-05-03T18:11:10.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.700+0000 7fafbdad4700 1 -- 172.21.15.122:0/963796855 wait complete. 2024-05-03T18:11:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.700+0000 7fafbdad4700 1 Processor -- start 2024-05-03T18:11:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.700+0000 7fafbdad4700 1 -- start start 2024-05-03T18:11:10.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.700+0000 7fafbdad4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb8102690 0x7fafb81040a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:10.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.700+0000 7fafbdad4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8103040 0x7fafb81045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:10.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafbdad4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb81076e0 0x7fafb8104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:10.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafbdad4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81125a0 con 0x7fafb8103040 2024-05-03T18:11:10.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafbdad4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8112420 con 0x7fafb81076e0 2024-05-03T18:11:10.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafbdad4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8112720 con 0x7fafb8102690 2024-05-03T18:11:10.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8103040 0x7fafb81045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:10.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8103040 0x7fafb81045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47988/0 (socket says 172.21.15.122:47988) 2024-05-03T18:11:10.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb6ffd700 1 -- 172.21.15.122:0/1329727875 learned_addr learned my addr 172.21.15.122:0/1329727875 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T18:11:10.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb77fe700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb8102690 0x7fafb81040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:10.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb7fff700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb81076e0 0x7fafb8104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:10.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb6ffd700 1 -- 172.21.15.122:0/1329727875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb8102690 msgr2=0x7fafb81040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:10.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb6ffd700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb8102690 0x7fafb81040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:10.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb6ffd700 1 -- 172.21.15.122:0/1329727875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb81076e0 msgr2=0x7fafb8104b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:10.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb6ffd700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb81076e0 0x7fafb8104b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:10.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb6ffd700 1 -- 172.21.15.122:0/1329727875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb81a8070 con 0x7fafb8103040 2024-05-03T18:11:10.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb6ffd700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8103040 0x7fafb81045e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fafac00e9f0 tx=0x7fafac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:10.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb4ff9700 1 -- 172.21.15.122:0/1329727875 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac00cbd0 con 0x7fafb8103040 2024-05-03T18:11:10.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb4ff9700 1 -- 172.21.15.122:0/1329727875 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafac00cd70 con 0x7fafb8103040 2024-05-03T18:11:10.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafbdad4700 1 -- 172.21.15.122:0/1329727875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb81a8360 con 0x7fafb8103040 2024-05-03T18:11:10.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafb4ff9700 1 -- 172.21.15.122:0/1329727875 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac005650 con 0x7fafb8103040 2024-05-03T18:11:10.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.701+0000 7fafbdad4700 1 -- 172.21.15.122:0/1329727875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb81a8850 con 0x7fafb8103040 2024-05-03T18:11:10.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.702+0000 7fafbdad4700 1 -- 172.21.15.122:0/1329727875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb806a190 con 0x7fafb8103040 2024-05-03T18:11:10.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.704+0000 7fafb4ff9700 1 -- 172.21.15.122:0/1329727875 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fafac004750 con 0x7fafb8103040 2024-05-03T18:11:10.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.704+0000 7fafb4ff9700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafa00b2910 0x7fafa00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:10.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.704+0000 7fafb77fe700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafa00b2910 0x7fafa00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:10.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.704+0000 7fafb4ff9700 1 -- 172.21.15.122:0/1329727875 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8153+0+0 (secure 0 0 0) 0x7fafac0e2720 con 0x7fafb8103040 2024-05-03T18:11:10.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.705+0000 7fafb77fe700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafa00b2910 0x7fafa00b4dd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fafa8005950 tx=0x7fafa800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:10.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.708+0000 7fafb4ff9700 1 -- 172.21.15.122:0/1329727875 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fafac054d70 con 0x7fafb8103040 2024-05-03T18:11:10.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:10 smithi149 ceph-mon[33409]: pgmap v844: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-03T18:11:10.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:10 smithi192 ceph-mon[33753]: pgmap v844: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-03T18:11:10.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.955+0000 7fafbdad4700 1 -- 172.21.15.122:0/1329727875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fafb802e510 con 0x7fafb8103040 2024-05-03T18:11:10.954 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:10 smithi122 ceph-mon[29744]: pgmap v844: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-03T18:11:10.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.957+0000 7fafb4ff9700 1 -- 172.21.15.122:0/1329727875 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7fafac0aa3e0 con 0x7fafb8103040 2024-05-03T18:11:10.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 -- 172.21.15.122:0/1329727875 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafa00b2910 msgr2=0x7fafa00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:10.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafa00b2910 0x7fafa00b4dd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fafa8005950 tx=0x7fafa800a300 comp rx=0 tx=0).stop 2024-05-03T18:11:10.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 -- 172.21.15.122:0/1329727875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8103040 msgr2=0x7fafb81045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:10.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8103040 0x7fafb81045e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fafac00e9f0 tx=0x7fafac00ed00 comp rx=0 tx=0).stop 2024-05-03T18:11:10.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 -- 172.21.15.122:0/1329727875 shutdown_connections 2024-05-03T18:11:10.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7fafa00b2910 0x7fafa00b4dd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:10.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb81076e0 0x7fafb8104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:10.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb8102690 0x7fafb81040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:10.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 --2- 172.21.15.122:0/1329727875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8103040 0x7fafb81045e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:10.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 -- 172.21.15.122:0/1329727875 >> 172.21.15.122:0/1329727875 conn(0x7fafb80783f0 msgr2=0x7fafb806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:10.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 -- 172.21.15.122:0/1329727875 shutdown_connections 2024-05-03T18:11:10.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:10.960+0000 7fafbdad4700 1 -- 172.21.15.122:0/1329727875 wait complete. 2024-05-03T18:11:10.960 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-03T18:11:11.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:10 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a[29740]: 2024-05-03T18:11:10.955+0000 7f04ca112700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T18:11:11.290 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 18:11:11 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b[63632]: ignoring --setuser ceph since I am not root 2024-05-03T18:11:11.291 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 18:11:11 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b[63632]: ignoring --setgroup ceph since I am not root 2024-05-03T18:11:11.291 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 18:11:11 smithi149 ceph-mds[63636]: -- 172.21.15.149:0/3973314299 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b7305bda40 con 0x55b730591000 2024-05-03T18:11:11.291 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 18:11:11 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b[63632]: starting mds.b at 2024-05-03T18:11:11.291 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 18:11:11 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h[63304]: ignoring --setuser ceph since I am not root 2024-05-03T18:11:11.291 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 18:11:11 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h[63304]: ignoring --setgroup ceph since I am not root 2024-05-03T18:11:11.291 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 18:11:11 smithi149 ceph-mds[63308]: -- 172.21.15.149:0/1309666352 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0bb729a40 con 0x55e0bb6fc800 2024-05-03T18:11:11.292 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 18:11:11 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h[63304]: starting mds.h at 2024-05-03T18:11:11.369 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c[64201]: ignoring --setuser ceph since I am not root 2024-05-03T18:11:11.369 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c[64201]: ignoring --setgroup ceph since I am not root 2024-05-03T18:11:11.369 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:11:11 smithi192 ceph-mds[64205]: -- 172.21.15.192:0/3081629662 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56374c4c5a40 con 0x56374c499000 2024-05-03T18:11:11.369 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c[64201]: starting mds.c at 2024-05-03T18:11:11.370 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i[64511]: ignoring --setuser ceph since I am not root 2024-05-03T18:11:11.370 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i[64511]: ignoring --setgroup ceph since I am not root 2024-05-03T18:11:11.370 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 18:11:11 smithi192 ceph-mds[64515]: -- 172.21.15.192:0/4117589508 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654f486da40 con 0x5654f4841000 2024-05-03T18:11:11.370 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i[64511]: starting mds.i at 2024-05-03T18:11:11.371 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f[63882]: ignoring --setuser ceph since I am not root 2024-05-03T18:11:11.371 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f[63882]: ignoring --setgroup ceph since I am not root 2024-05-03T18:11:11.371 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 18:11:11 smithi192 ceph-mds[63887]: -- 172.21.15.192:0/3568844564 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557906c7ba40 con 0x557906c4e800 2024-05-03T18:11:11.371 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f[63882]: starting mds.f at 2024-05-03T18:11:11.371 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l[63574]: ignoring --setuser ceph since I am not root 2024-05-03T18:11:11.372 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l[63574]: ignoring --setgroup ceph since I am not root 2024-05-03T18:11:11.372 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 18:11:11 smithi192 ceph-mds[63578]: -- 172.21.15.192:0/1135014874 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f4b6ada40 con 0x561f4b680800 2024-05-03T18:11:11.372 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 18:11:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l[63574]: starting mds.l at 2024-05-03T18:11:11.568 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:11 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1329727875' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T18:11:11.568 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:11 smithi122 ceph-mon[29744]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T18:11:11.568 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:11 smithi122 ceph-mon[29744]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T18:11:11.568 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:11 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/1329727875' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T18:11:11.569 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:11 smithi122 ceph-mon[29744]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T18:11:11.569 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:11 smithi122 ceph-mon[29744]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-03T18:11:11.597 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-03T18:11:11.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:11 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/1329727875' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T18:11:11.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:11 smithi149 ceph-mon[33409]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T18:11:11.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:11 smithi149 ceph-mon[33409]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T18:11:11.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:11 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/1329727875' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T18:11:11.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:11 smithi149 ceph-mon[33409]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T18:11:11.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:11 smithi149 ceph-mon[33409]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-03T18:11:11.838 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:11:11.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:11 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/1329727875' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T18:11:11.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:11 smithi192 ceph-mon[33753]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T18:11:11.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:11 smithi192 ceph-mon[33753]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T18:11:11.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:11 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/1329727875' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T18:11:11.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:11 smithi192 ceph-mon[33753]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T18:11:11.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:11 smithi192 ceph-mon[33753]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-03T18:11:12.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: pgmap v845: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-03T18:11:12.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.149:6836/1201077415,v1:172.21.15.149:6838/1201077415] up:boot 2024-05-03T18:11:12.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.149:6837/2170598566,v1:172.21.15.149:6839/2170598566] up:boot 2024-05-03T18:11:12.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.192:6832/3275231501,v1:172.21.15.192:6837/3275231501] up:boot 2024-05-03T18:11:12.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.192:6833/4265870039,v1:172.21.15.192:6836/4265870039] up:boot 2024-05-03T18:11:12.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.192:6834/716302496,v1:172.21.15.192:6838/716302496] up:boot 2024-05-03T18:11:12.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: mds.? [v2:172.21.15.192:6835/134876656,v1:172.21.15.192:6839/134876656] up:boot 2024-05-03T18:11:12.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-03T18:11:12.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:11:12.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:11:12.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:11:12.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:11:12.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:11:12.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:12 smithi149 ceph-mon[33409]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:11:12.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: pgmap v845: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-03T18:11:12.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.149:6836/1201077415,v1:172.21.15.149:6838/1201077415] up:boot 2024-05-03T18:11:12.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.149:6837/2170598566,v1:172.21.15.149:6839/2170598566] up:boot 2024-05-03T18:11:12.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.192:6832/3275231501,v1:172.21.15.192:6837/3275231501] up:boot 2024-05-03T18:11:12.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.192:6833/4265870039,v1:172.21.15.192:6836/4265870039] up:boot 2024-05-03T18:11:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.192:6834/716302496,v1:172.21.15.192:6838/716302496] up:boot 2024-05-03T18:11:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: mds.? [v2:172.21.15.192:6835/134876656,v1:172.21.15.192:6839/134876656] up:boot 2024-05-03T18:11:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-03T18:11:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:11:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:11:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:11:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:11:12.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:11:12.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:12 smithi122 ceph-mon[29744]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:11:12.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: pgmap v845: 97 pgs: 97 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-03T18:11:12.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.149:6836/1201077415,v1:172.21.15.149:6838/1201077415] up:boot 2024-05-03T18:11:12.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.149:6837/2170598566,v1:172.21.15.149:6839/2170598566] up:boot 2024-05-03T18:11:12.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.192:6832/3275231501,v1:172.21.15.192:6837/3275231501] up:boot 2024-05-03T18:11:12.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.192:6833/4265870039,v1:172.21.15.192:6836/4265870039] up:boot 2024-05-03T18:11:12.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.192:6834/716302496,v1:172.21.15.192:6838/716302496] up:boot 2024-05-03T18:11:12.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: mds.? [v2:172.21.15.192:6835/134876656,v1:172.21.15.192:6839/134876656] up:boot 2024-05-03T18:11:12.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-03T18:11:12.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:11:12.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:11:12.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:11:12.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:11:12.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:11:12.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:12 smithi192 ceph-mon[33753]: from='mgr.14152 172.21.15.122:0/1119058106' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:11:13.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.329+0000 7f5746a6c700 1 -- 172.21.15.122:0/3005489274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574006ba10 msgr2=0x7f574006be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:13.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.329+0000 7f5746a6c700 1 --2- 172.21.15.122:0/3005489274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574006ba10 0x7f574006be90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5734009990 tx=0x7f5734009ca0 comp rx=0 tx=0).stop 2024-05-03T18:11:13.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.329+0000 7f5746a6c700 1 -- 172.21.15.122:0/3005489274 shutdown_connections 2024-05-03T18:11:13.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.329+0000 7f5746a6c700 1 --2- 172.21.15.122:0/3005489274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f574006c3d0 0x7f5740110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:13.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.329+0000 7f5746a6c700 1 --2- 172.21.15.122:0/3005489274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740109500 0x7f57401098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:13.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.329+0000 7f5746a6c700 1 --2- 172.21.15.122:0/3005489274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574006ba10 0x7f574006be90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:13.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.329+0000 7f5746a6c700 1 -- 172.21.15.122:0/3005489274 >> 172.21.15.122:0/3005489274 conn(0x7f5740079310 msgr2=0x7f5740079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:13.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.329+0000 7f5746a6c700 1 -- 172.21.15.122:0/3005489274 shutdown_connections 2024-05-03T18:11:13.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.329+0000 7f5746a6c700 1 -- 172.21.15.122:0/3005489274 wait complete. 2024-05-03T18:11:13.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5746a6c700 1 Processor -- start 2024-05-03T18:11:13.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5746a6c700 1 -- start start 2024-05-03T18:11:13.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5746a6c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574006ba10 0x7f574019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:13.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5746a6c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574006c3d0 0x7f574019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:13.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5746a6c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5740109500 0x7f57401999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:13.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5746a6c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5740113630 con 0x7f574006c3d0 2024-05-03T18:11:13.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5746a6c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57401134b0 con 0x7f574006ba10 2024-05-03T18:11:13.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5746a6c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57401137b0 con 0x7f5740109500 2024-05-03T18:11:13.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5744808700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574006ba10 0x7f574019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:13.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5744808700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574006ba10 0x7f574019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.122:33550/0 (socket says 172.21.15.122:33550) 2024-05-03T18:11:13.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5744808700 1 -- 172.21.15.122:0/3652557380 learned_addr learned my addr 172.21.15.122:0/3652557380 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T18:11:13.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5745009700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5740109500 0x7f57401999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:13.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f573ffff700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574006c3d0 0x7f574019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:13.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5744808700 1 -- 172.21.15.122:0/3652557380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5740109500 msgr2=0x7f57401999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:13.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5744808700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5740109500 0x7f57401999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:13.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5744808700 1 -- 172.21.15.122:0/3652557380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574006c3d0 msgr2=0x7f574019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:13.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5744808700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574006c3d0 0x7f574019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:13.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f5744808700 1 -- 172.21.15.122:0/3652557380 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574019a150 con 0x7f574006ba10 2024-05-03T18:11:13.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.330+0000 7f573ffff700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574006c3d0 0x7f574019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:11:13.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.331+0000 7f5744808700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574006ba10 0x7f574019f830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f573800ef10 tx=0x7f573800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:13.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.331+0000 7f573dffb700 1 -- 172.21.15.122:0/3652557380 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573800c9c0 con 0x7f574006ba10 2024-05-03T18:11:13.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.331+0000 7f5746a6c700 1 -- 172.21.15.122:0/3652557380 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574019a3e0 con 0x7f574006ba10 2024-05-03T18:11:13.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.331+0000 7f573dffb700 1 -- 172.21.15.122:0/3652557380 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5738019070 con 0x7f574006ba10 2024-05-03T18:11:13.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.331+0000 7f573dffb700 1 -- 172.21.15.122:0/3652557380 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738014480 con 0x7f574006ba10 2024-05-03T18:11:13.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.331+0000 7f5746a6c700 1 -- 172.21.15.122:0/3652557380 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57401a6700 con 0x7f574006ba10 2024-05-03T18:11:13.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.332+0000 7f5746a6c700 1 -- 172.21.15.122:0/3652557380 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574006a190 con 0x7f574006ba10 2024-05-03T18:11:13.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.333+0000 7f573dffb700 1 -- 172.21.15.122:0/3652557380 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f5738012050 con 0x7f574006ba10 2024-05-03T18:11:13.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.334+0000 7f573dffb700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f57280b2910 0x7f57280b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:13.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.334+0000 7f573ffff700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f57280b2910 0x7f57280b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:13.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.334+0000 7f573dffb700 1 -- 172.21.15.122:0/3652557380 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8411+0+0 (secure 0 0 0) 0x7f57380e2a40 con 0x7f574006ba10 2024-05-03T18:11:13.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.334+0000 7f573ffff700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f57280b2910 0x7f57280b4dd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f573400b2e0 tx=0x7f573401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:13.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.338+0000 7f573dffb700 1 -- 172.21.15.122:0/3652557380 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5738010040 con 0x7f574006ba10 2024-05-03T18:11:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:13.589+0000 7f5746a6c700 1 -- 172.21.15.122:0/3652557380 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f574002e510 con 0x7f574006ba10 2024-05-03T18:11:14.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.487+0000 7f573dffb700 1 -- 172.21.15.122:0/3652557380 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f57380aa4f0 con 0x7f574006ba10 2024-05-03T18:11:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 -- 172.21.15.122:0/3652557380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f57280b2910 msgr2=0x7f57280b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f57280b2910 0x7f57280b4dd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f573400b2e0 tx=0x7f573401a040 comp rx=0 tx=0).stop 2024-05-03T18:11:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 -- 172.21.15.122:0/3652557380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574006ba10 msgr2=0x7f574019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574006ba10 0x7f574019f830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f573800ef10 tx=0x7f573800c3a0 comp rx=0 tx=0).stop 2024-05-03T18:11:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 -- 172.21.15.122:0/3652557380 shutdown_connections 2024-05-03T18:11:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f57280b2910 0x7f57280b4dd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5740109500 0x7f57401999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:14.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574006ba10 0x7f574019f830 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:14.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 --2- 172.21.15.122:0/3652557380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574006c3d0 0x7f574019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:14.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 -- 172.21.15.122:0/3652557380 >> 172.21.15.122:0/3652557380 conn(0x7f5740079310 msgr2=0x7f57401011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:14.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 -- 172.21.15.122:0/3652557380 shutdown_connections 2024-05-03T18:11:14.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:14.490+0000 7f5746a6c700 1 -- 172.21.15.122:0/3652557380 wait complete. 2024-05-03T18:11:14.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:14 smithi149 ceph-mon[33409]: pgmap v847: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 204 B/s wr, 5 op/s 2024-05-03T18:11:14.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:14 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3652557380' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:11:14.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:14 smithi149 ceph-mon[33409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:11:14.796 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:14 smithi122 ceph-mon[29744]: pgmap v847: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 204 B/s wr, 5 op/s 2024-05-03T18:11:14.796 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:14 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3652557380' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:11:14.796 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:14 smithi122 ceph-mon[29744]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:11:14.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:14 smithi192 ceph-mon[33753]: pgmap v847: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 204 B/s wr, 5 op/s 2024-05-03T18:11:14.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:14 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3652557380' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:11:14.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:14 smithi192 ceph-mon[33753]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:11:15.123 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-03T18:11:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:11:15.744 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:15 smithi122 ceph-mon[29744]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T18:11:15.744 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:15 smithi122 ceph-mon[29744]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T18:11:15.744 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:15 smithi122 ceph-mon[29744]: Cluster is now healthy 2024-05-03T18:11:15.744 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:15 smithi122 ceph-mon[29744]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:11:15.744 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:15 smithi122 ceph-mon[29744]: fsmap 12 up:standby 2024-05-03T18:11:15.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:15 smithi149 ceph-mon[33409]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T18:11:15.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:15 smithi149 ceph-mon[33409]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T18:11:15.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:15 smithi149 ceph-mon[33409]: Cluster is now healthy 2024-05-03T18:11:15.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:15 smithi149 ceph-mon[33409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:11:15.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:15 smithi149 ceph-mon[33409]: fsmap 12 up:standby 2024-05-03T18:11:15.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:15 smithi192 ceph-mon[33753]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T18:11:15.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:15 smithi192 ceph-mon[33753]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T18:11:15.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:15 smithi192 ceph-mon[33753]: Cluster is now healthy 2024-05-03T18:11:15.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:15 smithi192 ceph-mon[33753]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:11:15.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:15 smithi192 ceph-mon[33753]: fsmap 12 up:standby 2024-05-03T18:11:16.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:16 smithi149 ceph-mon[33409]: pgmap v848: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:11:16.814 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:16 smithi122 ceph-mon[29744]: pgmap v848: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:11:16.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:16 smithi192 ceph-mon[33753]: pgmap v848: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:11:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.065+0000 7f1c04acd700 1 -- 172.21.15.122:0/1283560317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c00102450 msgr2=0x7f1c00110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.065+0000 7f1c04acd700 1 --2- 172.21.15.122:0/1283560317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c00102450 0x7f1c00110340 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009990 tx=0x7f1bf8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.065+0000 7f1c04acd700 1 -- 172.21.15.122:0/1283560317 shutdown_connections 2024-05-03T18:11:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.065+0000 7f1c04acd700 1 --2- 172.21.15.122:0/1283560317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c00102450 0x7f1c00110340 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.065+0000 7f1c04acd700 1 --2- 172.21.15.122:0/1283560317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c00109520 0x7f1c00109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.065+0000 7f1c04acd700 1 --2- 172.21.15.122:0/1283560317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c00101a90 0x7f1c00101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.065+0000 7f1c04acd700 1 -- 172.21.15.122:0/1283560317 >> 172.21.15.122:0/1283560317 conn(0x7f1c00078a10 msgr2=0x7f1c000ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:17.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.065+0000 7f1c04acd700 1 -- 172.21.15.122:0/1283560317 shutdown_connections 2024-05-03T18:11:17.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.065+0000 7f1c04acd700 1 -- 172.21.15.122:0/1283560317 wait complete. 2024-05-03T18:11:17.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1c04acd700 1 Processor -- start 2024-05-03T18:11:17.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1c04acd700 1 -- start start 2024-05-03T18:11:17.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1c04acd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c00101a90 0x7f1c0019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:17.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1c04acd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c00102450 0x7f1c0019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:17.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1c04acd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c00109520 0x7f1c001998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:17.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1c04acd700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00113630 con 0x7f1c00101a90 2024-05-03T18:11:17.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1c04acd700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001134b0 con 0x7f1c00109520 2024-05-03T18:11:17.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1c04acd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001137b0 con 0x7f1c00102450 2024-05-03T18:11:17.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1bfe59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c00101a90 0x7f1c0019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:17.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1bfe59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c00101a90 0x7f1c0019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48016/0 (socket says 172.21.15.122:48016) 2024-05-03T18:11:17.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1bfe59c700 1 -- 172.21.15.122:0/3891897102 learned_addr learned my addr 172.21.15.122:0/3891897102 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T18:11:17.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.066+0000 7f1bfed9d700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c00109520 0x7f1c001998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:17.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1bfdd9b700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c00102450 0x7f1c0019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:17.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1bfe59c700 1 -- 172.21.15.122:0/3891897102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c00102450 msgr2=0x7f1c0019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:17.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1bfe59c700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c00102450 0x7f1c0019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1bfe59c700 1 -- 172.21.15.122:0/3891897102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c00109520 msgr2=0x7f1c001998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:17.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1bfe59c700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c00109520 0x7f1c001998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1bfe59c700 1 -- 172.21.15.122:0/3891897102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0019a030 con 0x7f1c00101a90 2024-05-03T18:11:17.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1bfdd9b700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c00102450 0x7f1c0019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:11:17.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1bfe59c700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c00101a90 0x7f1c0019f710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00060b0 tx=0x7f1bf000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:17.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1bf77fe700 1 -- 172.21.15.122:0/3891897102 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf001a070 con 0x7f1c00101a90 2024-05-03T18:11:17.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1bf77fe700 1 -- 172.21.15.122:0/3891897102 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bf000ddd0 con 0x7f1c00101a90 2024-05-03T18:11:17.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1c04acd700 1 -- 172.21.15.122:0/3891897102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0019a320 con 0x7f1c00101a90 2024-05-03T18:11:17.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1bf77fe700 1 -- 172.21.15.122:0/3891897102 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0014720 con 0x7f1c00101a90 2024-05-03T18:11:17.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.067+0000 7f1c04acd700 1 -- 172.21.15.122:0/3891897102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c001a65e0 con 0x7f1c00101a90 2024-05-03T18:11:17.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.068+0000 7f1c04acd700 1 -- 172.21.15.122:0/3891897102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0006a190 con 0x7f1c00101a90 2024-05-03T18:11:17.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.070+0000 7f1bf77fe700 1 -- 172.21.15.122:0/3891897102 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f1bf001d070 con 0x7f1c00101a90 2024-05-03T18:11:17.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.070+0000 7f1bf77fe700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bec0b2910 0x7f1bec0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:17.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.070+0000 7f1bfdd9b700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bec0b2910 0x7f1bec0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:17.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.070+0000 7f1bf77fe700 1 -- 172.21.15.122:0/3891897102 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8411+0+0 (secure 0 0 0) 0x7f1bf00e2070 con 0x7f1c00101a90 2024-05-03T18:11:17.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.071+0000 7f1bfdd9b700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bec0b2910 0x7f1bec0b4dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1be8005950 tx=0x7f1be80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:17.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.074+0000 7f1bf77fe700 1 -- 172.21.15.122:0/3891897102 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1bf0010070 con 0x7f1c00101a90 2024-05-03T18:11:17.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.277+0000 7f1c04acd700 1 -- 172.21.15.122:0/3891897102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1c0002e510 con 0x7f1c00101a90 2024-05-03T18:11:17.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.493+0000 7f1bf77fe700 1 -- 172.21.15.122:0/3891897102 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v81) ==== 181+0+0 (secure 0 0 0) 0x7f1bf00a9c20 con 0x7f1c00101a90 2024-05-03T18:11:17.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.496+0000 7f1c04acd700 1 -- 172.21.15.122:0/3891897102 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bec0b2910 msgr2=0x7f1bec0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:17.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.496+0000 7f1c04acd700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bec0b2910 0x7f1bec0b4dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1be8005950 tx=0x7f1be80058e0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.496+0000 7f1c04acd700 1 -- 172.21.15.122:0/3891897102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c00101a90 msgr2=0x7f1c0019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:17.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.496+0000 7f1c04acd700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c00101a90 0x7f1c0019f710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00060b0 tx=0x7f1bf000d3b0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.496+0000 7f1c04acd700 1 -- 172.21.15.122:0/3891897102 shutdown_connections 2024-05-03T18:11:17.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.496+0000 7f1c04acd700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f1bec0b2910 0x7f1bec0b4dd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.496+0000 7f1c04acd700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c00109520 0x7f1c001998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.496+0000 7f1c04acd700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c00101a90 0x7f1c0019f710 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.496+0000 7f1c04acd700 1 --2- 172.21.15.122:0/3891897102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c00102450 0x7f1c0019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:17.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.496+0000 7f1c04acd700 1 -- 172.21.15.122:0/3891897102 >> 172.21.15.122:0/3891897102 conn(0x7f1c00078a10 msgr2=0x7f1c00100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:17.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.496+0000 7f1c04acd700 1 -- 172.21.15.122:0/3891897102 shutdown_connections 2024-05-03T18:11:17.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:17.497+0000 7f1c04acd700 1 -- 172.21.15.122:0/3891897102 wait complete. 2024-05-03T18:11:17.496 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-05-03T18:11:17.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:17 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3891897102' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:11:17.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:17 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3891897102' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:11:17.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:17 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3891897102' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:11:18.190 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid a46fc846-0974-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-03T18:11:18.436 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:11:18.759 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:18 smithi122 ceph-mon[29744]: pgmap v849: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:11:18.760 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:18 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/3891897102' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:11:18.760 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:18 smithi122 ceph-mon[29744]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T18:11:18.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:18 smithi149 ceph-mon[33409]: pgmap v849: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:11:18.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:18 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/3891897102' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:11:18.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:18 smithi149 ceph-mon[33409]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T18:11:18.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:18 smithi192 ceph-mon[33753]: pgmap v849: 97 pgs: 97 active+clean; 640 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:11:18.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:18 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/3891897102' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:11:18.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:18 smithi192 ceph-mon[33753]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T18:11:20.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.116+0000 7f80abcf3700 1 -- 172.21.15.122:0/1946904159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a4101ed0 msgr2=0x7f80a4102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:20.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.116+0000 7f80abcf3700 1 --2- 172.21.15.122:0/1946904159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a4101ed0 0x7f80a4102350 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8094009990 tx=0x7f8094009ca0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.116+0000 7f80abcf3700 1 -- 172.21.15.122:0/1946904159 shutdown_connections 2024-05-03T18:11:20.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.116+0000 7f80abcf3700 1 --2- 172.21.15.122:0/1946904159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a4102890 0x7f80a4110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.116+0000 7f80abcf3700 1 --2- 172.21.15.122:0/1946904159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80a4109680 0x7f80a4109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.116+0000 7f80abcf3700 1 --2- 172.21.15.122:0/1946904159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a4101ed0 0x7f80a4102350 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.116+0000 7f80abcf3700 1 -- 172.21.15.122:0/1946904159 >> 172.21.15.122:0/1946904159 conn(0x7f80a4077610 msgr2=0x7f80a4077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:20.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.116+0000 7f80abcf3700 1 -- 172.21.15.122:0/1946904159 shutdown_connections 2024-05-03T18:11:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.116+0000 7f80abcf3700 1 -- 172.21.15.122:0/1946904159 wait complete. 2024-05-03T18:11:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.116+0000 7f80abcf3700 1 Processor -- start 2024-05-03T18:11:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.116+0000 7f80abcf3700 1 -- start start 2024-05-03T18:11:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80abcf3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a4101ed0 0x7f80a419be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80abcf3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a4102890 0x7f80a419c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80abcf3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80a4109680 0x7f80a41a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80abcf3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80a4113730 con 0x7f80a4101ed0 2024-05-03T18:11:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80abcf3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f80a41135b0 con 0x7f80a4102890 2024-05-03T18:11:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80abcf3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f80a41138b0 con 0x7f80a4109680 2024-05-03T18:11:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80a9a8f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a4101ed0 0x7f80a419be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80a9a8f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a4101ed0 0x7f80a419be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46486/0 (socket says 172.21.15.122:46486) 2024-05-03T18:11:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80a9a8f700 1 -- 172.21.15.122:0/2168884122 learned_addr learned my addr 172.21.15.122:0/2168884122 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-05-03T18:11:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80aa290700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80a4109680 0x7f80a41a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80a9a8f700 1 -- 172.21.15.122:0/2168884122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80a4109680 msgr2=0x7f80a41a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80a9a8f700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80a4109680 0x7f80a41a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80a9a8f700 1 -- 172.21.15.122:0/2168884122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a4102890 msgr2=0x7f80a419c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80a928e700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a4102890 0x7f80a419c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80a9a8f700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a4102890 0x7f80a419c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80a9a8f700 1 -- 172.21.15.122:0/2168884122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a41a0e10 con 0x7f80a4101ed0 2024-05-03T18:11:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80a928e700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a4102890 0x7f80a419c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:11:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80a9a8f700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a4101ed0 0x7f80a419be00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f80a000b530 tx=0x7f80a000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f809affd700 1 -- 172.21.15.122:0/2168884122 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0015020 con 0x7f80a4101ed0 2024-05-03T18:11:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f809affd700 1 -- 172.21.15.122:0/2168884122 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80a0011ce0 con 0x7f80a4101ed0 2024-05-03T18:11:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f809affd700 1 -- 172.21.15.122:0/2168884122 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0010d20 con 0x7f80a4101ed0 2024-05-03T18:11:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.117+0000 7f80abcf3700 1 -- 172.21.15.122:0/2168884122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a41a1100 con 0x7f80a4101ed0 2024-05-03T18:11:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.118+0000 7f80abcf3700 1 -- 172.21.15.122:0/2168884122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a41a8980 con 0x7f80a4101ed0 2024-05-03T18:11:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.119+0000 7f80abcf3700 1 -- 172.21.15.122:0/2168884122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a406a190 con 0x7f80a4101ed0 2024-05-03T18:11:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.120+0000 7f809affd700 1 -- 172.21.15.122:0/2168884122 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f80a0018020 con 0x7f80a4101ed0 2024-05-03T18:11:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.120+0000 7f809affd700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f80900b2910 0x7f80900b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.120+0000 7f80a928e700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f80900b2910 0x7f80900b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.120+0000 7f809affd700 1 -- 172.21.15.122:0/2168884122 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7915+0+0 (secure 0 0 0) 0x7f80a00e18e0 con 0x7f80a4101ed0 2024-05-03T18:11:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.121+0000 7f80a928e700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f80900b2910 0x7f80900b4dd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8094000c00 tx=0x7f8094005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.125+0000 7f809affd700 1 -- 172.21.15.122:0/2168884122 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f80a001f020 con 0x7f80a4101ed0 2024-05-03T18:11:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.329+0000 7f80abcf3700 1 -- 172.21.15.122:0/2168884122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f80a41a8c40 con 0x7f80a4101ed0 2024-05-03T18:11:20.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.497+0000 7f809affd700 1 -- 172.21.15.122:0/2168884122 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v82) ==== 169+0+0 (secure 0 0 0) 0x7f80a00a97a0 con 0x7f80a4101ed0 2024-05-03T18:11:20.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 -- 172.21.15.122:0/2168884122 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f80900b2910 msgr2=0x7f80900b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:20.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f80900b2910 0x7f80900b4dd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8094000c00 tx=0x7f8094005fb0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 -- 172.21.15.122:0/2168884122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a4101ed0 msgr2=0x7f80a419be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:20.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a4101ed0 0x7f80a419be00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f80a000b530 tx=0x7f80a000b8f0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 -- 172.21.15.122:0/2168884122 shutdown_connections 2024-05-03T18:11:20.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.122:6800/1089873263,v1:172.21.15.122:6801/1089873263] conn(0x7f80900b2910 0x7f80900b4dd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80a4109680 0x7f80a41a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a4101ed0 0x7f80a419be00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 --2- 172.21.15.122:0/2168884122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a4102890 0x7f80a419c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:20.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 -- 172.21.15.122:0/2168884122 >> 172.21.15.122:0/2168884122 conn(0x7f80a4077610 msgr2=0x7f80a41009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:20.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 -- 172.21.15.122:0/2168884122 shutdown_connections 2024-05-03T18:11:20.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-05-03T18:11:20.500+0000 7f80abcf3700 1 -- 172.21.15.122:0/2168884122 wait complete. 2024-05-03T18:11:20.501 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-05-03T18:11:20.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:20 smithi122 ceph-mon[29744]: pgmap v851: 65 pgs: 65 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:11:20.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:20 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2168884122' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:11:20.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:20 smithi149 ceph-mon[33409]: pgmap v851: 65 pgs: 65 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:11:20.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:20 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2168884122' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:11:20.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:20 smithi192 ceph-mon[33753]: pgmap v851: 65 pgs: 65 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:11:20.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:20 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2168884122' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:11:21.315 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:11:21.344 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:11:21.376 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:11:21.409 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-03T18:11:21.409 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T18:11:21.425 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T18:11:21.443 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T18:11:21.466 INFO:tasks.cephadm:Stopping all daemons... 2024-05-03T18:11:21.466 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-03T18:11:21.466 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.a 2024-05-03T18:11:21.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:21 smithi122 ceph-mon[29744]: from='client.? 172.21.15.122:0/2168884122' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:11:21.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:21 smithi122 ceph-mon[29744]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T18:11:21.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:21 smithi122 systemd[1]: Stopping Ceph mon.a for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:11:21.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:21 smithi149 ceph-mon[33409]: from='client.? 172.21.15.122:0/2168884122' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:11:21.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:21 smithi149 ceph-mon[33409]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T18:11:21.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:21 smithi192 ceph-mon[33753]: from='client.? 172.21.15.122:0/2168884122' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:11:21.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:21 smithi192 ceph-mon[33753]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T18:11:22.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:21 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a[29740]: 2024-05-03T18:11:21.788+0000 7f04cf91d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:11:22.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:21 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a[29740]: 2024-05-03T18:11:21.788+0000 7f04cf91d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-03T18:11:22.213 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:21 smithi122 podman[88144]: 2024-05-03 18:11:21.952441678 +0000 UTC m=+0.297943478 container died 6c70458e7d5bdd991afecfb7f7c6ee19b0d8e14070af64956377feb47a08e2ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-03T18:11:22.623 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:22 smithi122 podman[88144]: 2024-05-03 18:11:22.348967673 +0000 UTC m=+0.694469471 container cleanup 6c70458e7d5bdd991afecfb7f7c6ee19b0d8e14070af64956377feb47a08e2ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-03T18:11:22.624 INFO:journalctl@ceph.mon.a.smithi122.stdout:May 03 18:11:22 smithi122 bash[88144]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-a 2024-05-03T18:11:22.775 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.a.service' 2024-05-03T18:11:22.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:11:22.816 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-03T18:11:22.816 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-03T18:11:22.816 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.b 2024-05-03T18:11:23.107 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:22 smithi149 systemd[1]: Stopping Ceph mon.b for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:11:23.540 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:23 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-b[33405]: 2024-05-03T18:11:23.108+0000 7ff76f475700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:11:23.540 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:23 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-b[33405]: 2024-05-03T18:11:23.108+0000 7ff76f475700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-03T18:11:23.540 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:23 smithi149 podman[68179]: 2024-05-03 18:11:23.293292336 +0000 UTC m=+0.273174298 container died 35163dfde5309fdefb8bd8d985ced86364fc9aca9997600920f739ebf2761ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-03T18:11:24.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:23 smithi149 podman[68179]: 2024-05-03 18:11:23.827051515 +0000 UTC m=+0.806933518 container cleanup 35163dfde5309fdefb8bd8d985ced86364fc9aca9997600920f739ebf2761ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-03T18:11:24.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:May 03 18:11:23 smithi149 bash[68179]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-b 2024-05-03T18:11:24.247 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.b.service' 2024-05-03T18:11:24.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:11:24.288 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-03T18:11:24.288 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-03T18:11:24.289 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.c 2024-05-03T18:11:24.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:24 smithi192 systemd[1]: Stopping Ceph mon.c for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:11:25.067 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:24 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-c[33749]: 2024-05-03T18:11:24.649+0000 7f51205a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:11:25.067 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:24 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-c[33749]: 2024-05-03T18:11:24.649+0000 7f51205a5700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-03T18:11:25.067 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:24 smithi192 podman[69234]: 2024-05-03 18:11:24.793536348 +0000 UTC m=+0.304511446 container died cac55d07538833bcddc68a21dbcd5d36bcf32d849b27edd32e54362db352ab45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-c, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-03T18:11:25.369 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:25 smithi192 podman[69234]: 2024-05-03 18:11:25.284881613 +0000 UTC m=+0.795856709 container cleanup cac55d07538833bcddc68a21dbcd5d36bcf32d849b27edd32e54362db352ab45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-03T18:11:25.369 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 03 18:11:25 smithi192 bash[69234]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mon-c 2024-05-03T18:11:25.592 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mon.c.service' 2024-05-03T18:11:25.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:11:25.634 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-03T18:11:25.634 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-03T18:11:25.634 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.x 2024-05-03T18:11:25.963 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 18:11:25 smithi122 systemd[1]: Stopping Ceph mgr.x for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:11:26.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 18:11:25 smithi122 podman[88263]: 2024-05-03 18:11:25.99331822 +0000 UTC m=+0.155619612 container died fec6baba9e62d64a109cf367889b544c69d9c180975c19b11b3c41f72305da43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:11:26.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 18:11:26 smithi122 podman[88263]: 2024-05-03 18:11:26.430473189 +0000 UTC m=+0.592774581 container cleanup fec6baba9e62d64a109cf367889b544c69d9c180975c19b11b3c41f72305da43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240430) 2024-05-03T18:11:26.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:May 03 18:11:26 smithi122 bash[88263]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-x 2024-05-03T18:11:26.860 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-03T18:11:26.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:11:26.894 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-03T18:11:26.894 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-03T18:11:26.894 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.y 2024-05-03T18:11:27.234 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 18:11:26 smithi149 systemd[1]: Stopping Ceph mgr.y for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:11:27.539 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 18:11:27 smithi149 podman[68304]: 2024-05-03 18:11:27.236667782 +0000 UTC m=+0.144491182 container died 47a31b1263538a65b01fa25955cd33f701b7aaf9c20def3a7580b3585d41eb8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:11:27.790 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 18:11:27 smithi149 podman[68304]: 2024-05-03 18:11:27.624129082 +0000 UTC m=+0.531952479 container cleanup 47a31b1263538a65b01fa25955cd33f701b7aaf9c20def3a7580b3585d41eb8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:11:27.790 INFO:journalctl@ceph.mgr.y.smithi149.stdout:May 03 18:11:27 smithi149 bash[68304]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-y 2024-05-03T18:11:28.032 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-03T18:11:28.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:11:28.065 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-03T18:11:28.065 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-03T18:11:28.065 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.z 2024-05-03T18:11:28.369 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 18:11:28 smithi192 systemd[1]: Stopping Ceph mgr.z for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:11:28.689 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 18:11:28 smithi192 podman[69356]: 2024-05-03 18:11:28.393883307 +0000 UTC m=+0.139311945 container died a1483c20e32dc35b74312c92db8d909dbf503ee9d8d55e520d586c1f06df9cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:11:29.032 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 18:11:28 smithi192 podman[69356]: 2024-05-03 18:11:28.773924472 +0000 UTC m=+0.519353107 container cleanup a1483c20e32dc35b74312c92db8d909dbf503ee9d8d55e520d586c1f06df9cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-03T18:11:29.032 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 03 18:11:28 smithi192 bash[69356]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mgr-z 2024-05-03T18:11:29.179 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-03T18:11:29.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:11:29.212 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-03T18:11:29.212 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-03T18:11:29.212 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.0 2024-05-03T18:11:29.509 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:29 smithi122 systemd[1]: Stopping Ceph osd.0 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:11:29.963 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:29 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0[42145]: 2024-05-03T18:11:29.510+0000 7f8a525ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:11:29.963 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:29 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0[42145]: 2024-05-03T18:11:29.510+0000 7f8a525ce700 -1 osd.0 82 *** Got signal Terminated *** 2024-05-03T18:11:29.964 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:29 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0[42145]: 2024-05-03T18:11:29.510+0000 7f8a525ce700 -1 osd.0 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:11:35.213 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:34 smithi122 podman[88385]: 2024-05-03 18:11:34.946890373 +0000 UTC m=+5.549812743 container died 209ca2f75e9340b127d3d7f36de3f9892392858aaf50dbd945fec8fe6ce684b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:11:35.603 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:35 smithi122 podman[88385]: 2024-05-03 18:11:35.35349341 +0000 UTC m=+5.956415781 container cleanup 209ca2f75e9340b127d3d7f36de3f9892392858aaf50dbd945fec8fe6ce684b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:11:35.603 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:35 smithi122 bash[88385]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0 2024-05-03T18:11:35.963 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:35 smithi122 podman[88402]: 2024-05-03 18:11:35.605584676 +0000 UTC m=+0.655408793 container remove 209ca2f75e9340b127d3d7f36de3f9892392858aaf50dbd945fec8fe6ce684b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-03T18:11:36.358 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:36 smithi122 podman[88495]: 2024-05-03 18:11:35.931915718 +0000 UTC m=+0.017976105 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:11:36.358 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:36 smithi122 podman[88495]: 2024-05-03 18:11:36.039977308 +0000 UTC m=+0.126037682 container create 823bbca134f7a67c303bf2a79101056acd76db284d25db5c2be860026714bcd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:11:36.358 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:36 smithi122 podman[88495]: 2024-05-03 18:11:36.259513639 +0000 UTC m=+0.345574014 container init 823bbca134f7a67c303bf2a79101056acd76db284d25db5c2be860026714bcd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-03T18:11:36.358 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:36 smithi122 podman[88495]: 2024-05-03 18:11:36.262531951 +0000 UTC m=+0.348592326 container start 823bbca134f7a67c303bf2a79101056acd76db284d25db5c2be860026714bcd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:11:36.358 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:36 smithi122 podman[88495]: 2024-05-03 18:11:36.359660968 +0000 UTC m=+0.445721352 container attach 823bbca134f7a67c303bf2a79101056acd76db284d25db5c2be860026714bcd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430) 2024-05-03T18:11:36.713 INFO:journalctl@ceph.osd.0.smithi122.stdout:May 03 18:11:36 smithi122 podman[88495]: 2024-05-03 18:11:36.559255184 +0000 UTC m=+0.645315560 container died 823bbca134f7a67c303bf2a79101056acd76db284d25db5c2be860026714bcd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-03T18:11:37.233 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.0.service' 2024-05-03T18:11:37.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:11:37.272 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-03T18:11:37.272 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-03T18:11:37.272 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.1 2024-05-03T18:11:37.586 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:37 smithi149 systemd[1]: Stopping Ceph osd.1 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:11:38.040 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:37 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1[38021]: 2024-05-03T18:11:37.587+0000 7f9a65f08700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:11:38.040 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:37 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1[38021]: 2024-05-03T18:11:37.587+0000 7f9a65f08700 -1 osd.1 82 *** Got signal Terminated *** 2024-05-03T18:11:38.041 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:37 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1[38021]: 2024-05-03T18:11:37.587+0000 7f9a65f08700 -1 osd.1 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:11:43.263 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:42 smithi149 podman[68429]: 2024-05-03 18:11:42.953273305 +0000 UTC m=+5.489918048 container died 44041540fe9fa2ae2506813e63f5803280b51441a8e0365b3d429a49bd87d364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-05-03T18:11:43.540 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:43 smithi149 podman[68429]: 2024-05-03 18:11:43.364457349 +0000 UTC m=+5.901102145 container cleanup 44041540fe9fa2ae2506813e63f5803280b51441a8e0365b3d429a49bd87d364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-03T18:11:43.540 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:43 smithi149 bash[68429]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1 2024-05-03T18:11:44.033 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:43 smithi149 podman[68446]: 2024-05-03 18:11:43.590658225 +0000 UTC m=+0.639967641 container remove 44041540fe9fa2ae2506813e63f5803280b51441a8e0365b3d429a49bd87d364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:11:44.290 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:44 smithi149 podman[68539]: 2024-05-03 18:11:43.939121765 +0000 UTC m=+0.018480195 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:11:44.291 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:44 smithi149 podman[68539]: 2024-05-03 18:11:44.074917812 +0000 UTC m=+0.154276218 container create 9bf13ae19a433afd82634c8eeb109aa233ebc5f280d87cb091e93062525104c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-03T18:11:44.650 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:44 smithi149 podman[68539]: 2024-05-03 18:11:44.353283225 +0000 UTC m=+0.432641637 container init 9bf13ae19a433afd82634c8eeb109aa233ebc5f280d87cb091e93062525104c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:11:44.651 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:44 smithi149 podman[68539]: 2024-05-03 18:11:44.356553833 +0000 UTC m=+0.435912241 container start 9bf13ae19a433afd82634c8eeb109aa233ebc5f280d87cb091e93062525104c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-05-03T18:11:44.651 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:44 smithi149 podman[68539]: 2024-05-03 18:11:44.445350705 +0000 UTC m=+0.524709130 container attach 9bf13ae19a433afd82634c8eeb109aa233ebc5f280d87cb091e93062525104c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:11:44.996 INFO:journalctl@ceph.osd.1.smithi149.stdout:May 03 18:11:44 smithi149 podman[68539]: 2024-05-03 18:11:44.653160627 +0000 UTC m=+0.732519052 container died 9bf13ae19a433afd82634c8eeb109aa233ebc5f280d87cb091e93062525104c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-03T18:11:45.300 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.1.service' 2024-05-03T18:11:45.341 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:11:45.341 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-03T18:11:45.342 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-03T18:11:45.342 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.2 2024-05-03T18:11:45.619 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:45 smithi192 systemd[1]: Stopping Ceph osd.2 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:11:45.879 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:45 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2[38922]: 2024-05-03T18:11:45.727+0000 7f156e19a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:11:45.879 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:45 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2[38922]: 2024-05-03T18:11:45.727+0000 7f156e19a700 -1 osd.2 82 *** Got signal Terminated *** 2024-05-03T18:11:45.879 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:45 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2[38922]: 2024-05-03T18:11:45.727+0000 7f156e19a700 -1 osd.2 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:11:51.346 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:51 smithi192 podman[69480]: 2024-05-03 18:11:51.05879948 +0000 UTC m=+5.518831489 container died e41c69e15eae71295c132ac7a29026ecb55ca1462f06f0f7c155d107c376e37f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-03T18:11:51.619 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:51 smithi192 podman[69480]: 2024-05-03 18:11:51.437197557 +0000 UTC m=+5.897229557 container cleanup e41c69e15eae71295c132ac7a29026ecb55ca1462f06f0f7c155d107c376e37f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2) 2024-05-03T18:11:51.619 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:51 smithi192 bash[69480]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2 2024-05-03T18:11:52.053 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:51 smithi192 podman[69509]: 2024-05-03 18:11:51.621893281 +0000 UTC m=+0.563886878 container remove e41c69e15eae71295c132ac7a29026ecb55ca1462f06f0f7c155d107c376e37f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-05-03T18:11:52.365 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:52 smithi192 podman[69602]: 2024-05-03 18:11:51.958770467 +0000 UTC m=+0.017686651 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:11:52.366 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:52 smithi192 podman[69602]: 2024-05-03 18:11:52.139992618 +0000 UTC m=+0.198908781 container create 0facc50c31463acfd14100312c0153692783b02048fcd2044ec7c873169ab3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240430, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-03T18:11:52.619 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:52 smithi192 podman[69602]: 2024-05-03 18:11:52.367638354 +0000 UTC m=+0.426554551 container init 0facc50c31463acfd14100312c0153692783b02048fcd2044ec7c873169ab3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-03T18:11:52.620 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:52 smithi192 podman[69602]: 2024-05-03 18:11:52.371112494 +0000 UTC m=+0.430028674 container start 0facc50c31463acfd14100312c0153692783b02048fcd2044ec7c873169ab3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-03T18:11:52.620 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:52 smithi192 podman[69602]: 2024-05-03 18:11:52.49285418 +0000 UTC m=+0.551770361 container attach 0facc50c31463acfd14100312c0153692783b02048fcd2044ec7c873169ab3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240430, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:11:53.036 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 03 18:11:52 smithi192 podman[69602]: 2024-05-03 18:11:52.676717124 +0000 UTC m=+0.735633306 container died 0facc50c31463acfd14100312c0153692783b02048fcd2044ec7c873169ab3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:11:53.335 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.2.service' 2024-05-03T18:11:53.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:11:53.375 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-03T18:11:53.375 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-03T18:11:53.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.3 2024-05-03T18:11:53.700 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:53 smithi122 systemd[1]: Stopping Ceph osd.3 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:11:53.963 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:53 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3[48823]: 2024-05-03T18:11:53.701+0000 7ff6f026e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:11:53.963 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:53 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3[48823]: 2024-05-03T18:11:53.701+0000 7ff6f026e700 -1 osd.3 82 *** Got signal Terminated *** 2024-05-03T18:11:53.963 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:53 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3[48823]: 2024-05-03T18:11:53.701+0000 7ff6f026e700 -1 osd.3 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:11:55.713 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:55 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3[48823]: 2024-05-03T18:11:55.459+0000 7ff6ec862700 -1 osd.3 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:29.376754+0000 front 2024-05-03T18:11:29.376773+0000 (oldest deadline 2024-05-03T18:11:55.276799+0000) 2024-05-03T18:11:56.119 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:11:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:11:56.017+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:11:56.119 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:11:55 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:11:55.854+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:11:56.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:11:55 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:11:55.882+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:11:56.713 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:56 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3[48823]: 2024-05-03T18:11:56.426+0000 7ff6ec862700 -1 osd.3 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:29.376754+0000 front 2024-05-03T18:11:29.376773+0000 (oldest deadline 2024-05-03T18:11:55.276799+0000) 2024-05-03T18:11:57.093 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:11:56 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:11:56.813+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:11:57.119 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:11:57 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:11:57.049+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:11:57.119 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:11:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:11:56.808+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:11:57.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:11:56 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:11:56.836+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:11:57.713 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3[48823]: 2024-05-03T18:11:57.442+0000 7ff6ec862700 -1 osd.3 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:29.376754+0000 front 2024-05-03T18:11:29.376773+0000 (oldest deadline 2024-05-03T18:11:55.276799+0000) 2024-05-03T18:11:58.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:11:57 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:11:57.766+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:11:58.119 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:11:58 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:11:58.071+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:11:58.119 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:11:57 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:11:57.855+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:11:58.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:11:57 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:11:57.814+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:11:58.801 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:58 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3[48823]: 2024-05-03T18:11:58.465+0000 7ff6ec862700 -1 osd.3 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:29.376754+0000 front 2024-05-03T18:11:29.376773+0000 (oldest deadline 2024-05-03T18:11:55.276799+0000) 2024-05-03T18:11:58.802 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:11:58 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:11:58.528+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:11:59.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:11:58 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:11:58.724+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:11:59.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:11:58 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:11:58.803+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:11:59.214 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:59 smithi122 podman[88616]: 2024-05-03 18:11:59.024788687 +0000 UTC m=+5.444011083 container died d05a30e9caf07815f5c5a8a7114b7b3ed8c21ad66771f0c4db6bcace43b6ea34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:11:59.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:11:59 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:11:59.106+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:11:59.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:11:58 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:11:58.901+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:11:59.713 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:11:59 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:11:59.549+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:11:59.713 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:59 smithi122 podman[88616]: 2024-05-03 18:11:59.409893118 +0000 UTC m=+5.829115522 container cleanup d05a30e9caf07815f5c5a8a7114b7b3ed8c21ad66771f0c4db6bcace43b6ea34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:11:59.714 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:59 smithi122 bash[88616]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3 2024-05-03T18:11:59.714 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:59 smithi122 podman[88633]: 2024-05-03 18:11:59.56181472 +0000 UTC m=+0.529283533 container remove d05a30e9caf07815f5c5a8a7114b7b3ed8c21ad66771f0c4db6bcace43b6ea34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-03T18:12:00.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:11:59 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:11:59.730+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:00.107 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:11:59 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:11:59.818+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:00.108 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:59 smithi122 podman[88723]: 2024-05-03 18:11:59.875376621 +0000 UTC m=+0.018179339 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:12:00.108 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:11:59 smithi122 podman[88723]: 2024-05-03 18:11:59.988005173 +0000 UTC m=+0.130807876 container create fd7ca2b9b424476687edac3508e3736cf37a4cdca365d6c51864bd97d3a8aad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3-deactivate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-05-03T18:12:00.350 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:00 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:00.080+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:00.350 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:11:59 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:11:59.951+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:00.463 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:12:00 smithi122 podman[88723]: 2024-05-03 18:12:00.224521199 +0000 UTC m=+0.367323903 container init fd7ca2b9b424476687edac3508e3736cf37a4cdca365d6c51864bd97d3a8aad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-03T18:12:00.463 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:12:00 smithi122 podman[88723]: 2024-05-03 18:12:00.227584652 +0000 UTC m=+0.370387356 container start fd7ca2b9b424476687edac3508e3736cf37a4cdca365d6c51864bd97d3a8aad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:12:00.463 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:12:00 smithi122 podman[88723]: 2024-05-03 18:12:00.316114081 +0000 UTC m=+0.458916788 container attach fd7ca2b9b424476687edac3508e3736cf37a4cdca365d6c51864bd97d3a8aad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-03T18:12:00.619 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:00 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:00.350+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:00.827 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:00 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:00.585+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:00.827 INFO:journalctl@ceph.osd.3.smithi122.stdout:May 03 18:12:00 smithi122 podman[88723]: 2024-05-03 18:12:00.516539554 +0000 UTC m=+0.659342259 container died fd7ca2b9b424476687edac3508e3736cf37a4cdca365d6c51864bd97d3a8aad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:12:01.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:00 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:00.690+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:01.101 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:00 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:00.857+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:01.121 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.3.service' 2024-05-03T18:12:01.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:12:01.162 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-03T18:12:01.162 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-03T18:12:01.162 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.4 2024-05-03T18:12:01.301 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:01 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:01.129+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:01.302 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:00 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:00.910+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:01.484 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:01 smithi149 systemd[1]: Stopping Ceph osd.4 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:12:01.619 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:01 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:01.303+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:01.790 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:01 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:01.687+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:01.790 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:01 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4[44962]: 2024-05-03T18:12:01.485+0000 7f9f4695c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:12:01.791 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:01 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4[44962]: 2024-05-03T18:12:01.485+0000 7f9f4695c700 -1 osd.4 82 *** Got signal Terminated *** 2024-05-03T18:12:01.791 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:01 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4[44962]: 2024-05-03T18:12:01.485+0000 7f9f4695c700 -1 osd.4 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:12:01.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:01 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:01.610+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:01.963 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:01 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:01.813+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:02.338 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:02 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:02.091+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:02.339 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:01 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:01.947+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:02.619 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:02 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:02.339+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:02.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:02 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:02.597+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:02.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:02 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:02.826+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:03.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:02 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:02.691+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:03.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:03 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:03.072+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:03.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:02 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:02.903+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:03.540 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:03 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:03.259+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:03.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:03 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:03.382+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:03.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:03 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:03.637+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:03.963 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:03 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:03.797+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:04.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:03 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:03.658+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:04.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:04 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:04.035+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:04.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:03 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:03.886+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:04.674 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:04 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:04.306+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:04.844 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:04 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:04.393+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:04.844 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:04 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:04.393+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.945142+0000 front 2024-05-03T18:11:39.945124+0000 (oldest deadline 2024-05-03T18:12:04.045126+0000) 2024-05-03T18:12:04.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:04 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:04.673+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:04.963 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:04 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:04.788+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:04.963 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:04 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:04.788+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:05.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:04 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:04.675+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:05.119 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:05 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:05.042+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:05.119 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:04 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:04.846+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:05.709 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:05 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:05.322+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:05.862 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:05 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:05.424+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:05.863 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:05 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:05.424+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.945142+0000 front 2024-05-03T18:11:39.945124+0000 (oldest deadline 2024-05-03T18:12:04.045126+0000) 2024-05-03T18:12:05.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:05 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:05.675+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:05.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:05 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:05.675+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:05.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:05 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:05.759+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:05.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:05 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:05.759+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:06.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:05 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:05.710+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:06.119 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:06 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:06.043+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:06.119 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:05 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:05.864+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:06.120 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:05 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:05.864+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:06.727 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:06 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:06.329+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:06.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:06 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:06.465+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:06.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:06 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:06.465+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.945142+0000 front 2024-05-03T18:11:39.945124+0000 (oldest deadline 2024-05-03T18:12:04.045126+0000) 2024-05-03T18:12:06.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:06 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:06.699+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:06.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:06 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:06.699+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:06.963 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:06 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:06.718+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:06.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:06 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:06.718+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:07.040 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:06 smithi149 podman[68656]: 2024-05-03 18:12:06.826366823 +0000 UTC m=+5.462875288 container died f658c8da154a9edb88b53894963de3d1a7778de6787fe6f10a746608fb241e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-03T18:12:07.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:06 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:06.728+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:07.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:07 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:07.049+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:07.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:06 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:06.904+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:07.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:06 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:06.904+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:07.460 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:07 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:07.283+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:07.460 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:07 smithi149 podman[68656]: 2024-05-03 18:12:07.194120291 +0000 UTC m=+5.830628768 container cleanup f658c8da154a9edb88b53894963de3d1a7778de6787fe6f10a746608fb241e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-03T18:12:07.461 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:07 smithi149 bash[68656]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4 2024-05-03T18:12:07.790 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:07 smithi149 podman[68673]: 2024-05-03 18:12:07.462610583 +0000 UTC m=+0.638640626 container remove f658c8da154a9edb88b53894963de3d1a7778de6787fe6f10a746608fb241e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-03T18:12:07.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:07 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:07.681+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:07.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:07 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:07.480+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:07.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:07 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:07.480+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.945142+0000 front 2024-05-03T18:11:39.945124+0000 (oldest deadline 2024-05-03T18:12:04.045126+0000) 2024-05-03T18:12:07.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:07 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:07.718+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:07.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:07 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:07.718+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:07.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:07 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:07.713+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:07.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:07 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:07.713+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:08.106 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:07 smithi149 podman[68766]: 2024-05-03 18:12:07.710207255 +0000 UTC m=+0.018211915 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:12:08.131 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:07 smithi149 podman[68766]: 2024-05-03 18:12:07.813480842 +0000 UTC m=+0.121485526 container create dcc84d6362897ee01b0a07133192e7ad8edcad91c7fa9066a02f9b6bc3fd35c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-05-03T18:12:08.132 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:08 smithi149 podman[68766]: 2024-05-03 18:12:08.041063524 +0000 UTC m=+0.349068165 container init dcc84d6362897ee01b0a07133192e7ad8edcad91c7fa9066a02f9b6bc3fd35c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD) 2024-05-03T18:12:08.132 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:08 smithi149 podman[68766]: 2024-05-03 18:12:08.044203867 +0000 UTC m=+0.352208509 container start dcc84d6362897ee01b0a07133192e7ad8edcad91c7fa9066a02f9b6bc3fd35c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-03T18:12:08.132 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:08 smithi149 podman[68766]: 2024-05-03 18:12:08.107995565 +0000 UTC m=+0.416000207 container attach dcc84d6362897ee01b0a07133192e7ad8edcad91c7fa9066a02f9b6bc3fd35c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-03T18:12:08.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:08 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:08.094+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:08.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:07 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:07.889+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:08.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:07 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:07.889+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:08.540 INFO:journalctl@ceph.osd.4.smithi149.stdout:May 03 18:12:08 smithi149 podman[68766]: 2024-05-03 18:12:08.341294959 +0000 UTC m=+0.649299608 container died dcc84d6362897ee01b0a07133192e7ad8edcad91c7fa9066a02f9b6bc3fd35c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-03T18:12:08.541 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:08 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:08.288+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:08.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:08 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:08.488+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:08.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:08 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:08.488+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.945142+0000 front 2024-05-03T18:11:39.945124+0000 (oldest deadline 2024-05-03T18:12:04.045126+0000) 2024-05-03T18:12:08.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:08 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:08.758+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:08.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:08 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:08.758+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:08.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:08 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:08.679+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:08.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:08 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:08.679+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:09.010 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:08 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:08.708+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:09.010 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:08 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:08.708+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:09.027 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.4.service' 2024-05-03T18:12:09.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:12:09.065 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-03T18:12:09.065 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-03T18:12:09.065 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.5 2024-05-03T18:12:09.369 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:09 smithi192 systemd[1]: Stopping Ceph osd.5 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:12:09.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:09 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:09.085+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:09.371 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:08 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:08.903+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:09.371 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:08 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:08.903+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:09.710 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:09 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:09.329+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:09.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:09 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:09.374+0000 7f610181e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:12:09.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:09 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:09.374+0000 7f610181e700 -1 osd.5 82 *** Got signal Terminated *** 2024-05-03T18:12:09.870 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:09 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:09.374+0000 7f610181e700 -1 osd.5 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:12:09.870 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:09 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:09.519+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:09.870 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:09 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:09.519+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.945142+0000 front 2024-05-03T18:11:39.945124+0000 (oldest deadline 2024-05-03T18:12:04.045126+0000) 2024-05-03T18:12:09.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:09 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:09.775+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:09.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:09 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:09.775+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:09.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:09 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:09.666+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:09.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:09 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:09.666+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:10.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:09 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:09.711+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:10.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:09 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:09.711+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:10.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:10 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:10.082+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:10.371 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:09 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:09.939+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:10.371 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:09 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:09.939+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:10.666 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:10 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:10.296+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:10.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:10 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:10.497+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:10.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:10 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:10.497+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.945142+0000 front 2024-05-03T18:11:39.945124+0000 (oldest deadline 2024-05-03T18:12:04.045126+0000) 2024-05-03T18:12:10.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:10 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:10.760+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:10.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:10 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:10.760+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:10.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:10 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:10.683+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:10.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:10 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:10.683+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:11.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:10 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:10.667+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:11.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:10 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:10.667+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:11.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:11.130+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:11.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:10 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:10.944+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:11.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:10 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:10.944+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:11.650 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:11 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:11.305+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:11.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:11.521+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:11.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:11.521+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.945142+0000 front 2024-05-03T18:11:39.945124+0000 (oldest deadline 2024-05-03T18:12:04.045126+0000) 2024-05-03T18:12:11.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:11 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:11.729+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:11.963 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:11 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:11.729+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:11.963 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:11 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:11.654+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:11.964 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:11 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:11.654+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:12.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:11 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:11.652+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:12.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:11 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:11.652+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:12.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:11 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:11.652+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:12.163 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:11.906+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:12.164 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:11 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:11.906+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:12.527 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:12 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:12.165+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:12.674 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:12 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:12.292+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:12.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:12 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:12.528+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:12.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:12 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:12.528+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.945142+0000 front 2024-05-03T18:11:39.945124+0000 (oldest deadline 2024-05-03T18:12:04.045126+0000) 2024-05-03T18:12:12.963 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:12 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:12.702+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:12.963 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:12 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:12.702+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:12.964 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:12 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:12.714+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:12.964 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:12 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:12.714+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:13.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:12 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:12.675+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:13.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:12 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:12.675+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:13.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:12 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:12.675+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:13.186 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:12 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:12.871+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:13.187 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:12 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:12.871+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:13.553 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:13 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:13.187+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:13.553 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:13 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:13.187+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:13.689 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:13 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:13.320+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:13.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:13 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:13.554+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:28.744114+0000 front 2024-05-03T18:11:28.744010+0000 (oldest deadline 2024-05-03T18:11:59.944627+0000) 2024-05-03T18:12:13.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:13 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:13.554+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.945142+0000 front 2024-05-03T18:11:39.945124+0000 (oldest deadline 2024-05-03T18:12:04.045126+0000) 2024-05-03T18:12:13.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:13 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5[45723]: 2024-05-03T18:12:13.554+0000 7f60f9e0a700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.346201+0000 front 2024-05-03T18:11:49.346146+0000 (oldest deadline 2024-05-03T18:12:12.746798+0000) 2024-05-03T18:12:14.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:13 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:13.690+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:14.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:13 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:13.690+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:14.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:13 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:13.690+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:14.163 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:13 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:13.899+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:14.163 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:13 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:13.899+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:14.164 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:13 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:13.899+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:14.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:13 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:13.758+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:14.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:13 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:13.758+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:14.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:13 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:13.748+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:14.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:13 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:13.748+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:14.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:13 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:13.748+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:14.540 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:14 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:14.285+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:14.619 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:14 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:14.165+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:14.619 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:14 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:14.165+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:15.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:14 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:14.655+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:15.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:14 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:14.655+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:15.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:14 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:14.655+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:15.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:14 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:14.757+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:15.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:14 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:14.757+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:15.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:14 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:14.756+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:15.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:14 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:14.756+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:15.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:14 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:14.756+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:15.224 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:15 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:15.118+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:15.224 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:15 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:15.118+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:15.224 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:14 smithi192 podman[69709]: 2024-05-03 18:12:14.904357217 +0000 UTC m=+5.647073212 container died d8886656ad3d1645ada301c0ef419605fc590d29cbf8b0b9cdfb71d62007c0be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:12:15.224 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:14 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:14.901+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:15.224 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:14 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:14.901+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:15.225 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:14 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:14.901+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:15.525 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:15 smithi192 podman[69709]: 2024-05-03 18:12:15.342219546 +0000 UTC m=+6.084935546 container cleanup d8886656ad3d1645ada301c0ef419605fc590d29cbf8b0b9cdfb71d62007c0be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-03T18:12:15.526 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:15 smithi192 bash[69709]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5 2024-05-03T18:12:15.676 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:15 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:15.332+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:15.676 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:15 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:15.332+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:15.869 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:15 smithi192 podman[69726]: 2024-05-03 18:12:15.5275952 +0000 UTC m=+0.626328458 container remove d8886656ad3d1645ada301c0ef419605fc590d29cbf8b0b9cdfb71d62007c0be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:12:16.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:15 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:15.677+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:16.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:15 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:15.677+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:16.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:15 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:15.677+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:16.188 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:16 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:16.078+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:16.189 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:16 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:16.078+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:16.189 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:15 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:15.896+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:16.189 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:15 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:15.896+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:16.189 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:15 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:15.896+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:16.190 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:15 smithi192 podman[69817]: 2024-05-03 18:12:15.953542547 +0000 UTC m=+0.113294528 container create 2c64a347d1f5a021cc8adc00218aab28f78026e45a0aa8d5b07e49c99e113c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:12:16.190 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:15 smithi192 podman[69817]: 2024-05-03 18:12:15.858422149 +0000 UTC m=+0.018174129 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:12:16.190 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:16 smithi192 podman[69817]: 2024-05-03 18:12:16.19098553 +0000 UTC m=+0.350737516 container init 2c64a347d1f5a021cc8adc00218aab28f78026e45a0aa8d5b07e49c99e113c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-03T18:12:16.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:15 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:15.786+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:16.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:15 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:15.786+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:16.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:15 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:15.786+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.699934+0000 front 2024-05-03T18:11:50.699847+0000 (oldest deadline 2024-05-03T18:12:15.399758+0000) 2024-05-03T18:12:16.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:15 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:15.802+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:16.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:15 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:15.802+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:16.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:15 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:15.802+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:16.489 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:16 smithi192 podman[69817]: 2024-05-03 18:12:16.194415269 +0000 UTC m=+0.354167248 container start 2c64a347d1f5a021cc8adc00218aab28f78026e45a0aa8d5b07e49c99e113c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-03T18:12:16.490 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:16 smithi192 podman[69817]: 2024-05-03 18:12:16.290681359 +0000 UTC m=+0.450433350 container attach 2c64a347d1f5a021cc8adc00218aab28f78026e45a0aa8d5b07e49c99e113c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-03T18:12:16.678 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:16 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:16.381+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:16.678 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:16 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:16.381+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:16.808 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 03 18:12:16 smithi192 podman[69817]: 2024-05-03 18:12:16.492064637 +0000 UTC m=+0.651816624 container died 2c64a347d1f5a021cc8adc00218aab28f78026e45a0aa8d5b07e49c99e113c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:12:17.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:16 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:16.679+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:17.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:16 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:16.679+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:17.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:16 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:16.679+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:17.081 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:16 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:16.922+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:17.081 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:16 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:16.922+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:17.081 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:16 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:16.922+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:17.108 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.5.service' 2024-05-03T18:12:17.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:12:17.146 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-03T18:12:17.146 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-03T18:12:17.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.6 2024-05-03T18:12:17.172 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:16.796+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:17.172 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:16.796+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:17.173 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:16.796+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:17.173 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:16.827+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:17.173 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:16.827+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:17.173 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:16 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:16.827+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.699934+0000 front 2024-05-03T18:11:50.699847+0000 (oldest deadline 2024-05-03T18:12:15.399758+0000) 2024-05-03T18:12:17.360 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:17 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:17.083+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:17.360 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:17 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:17.083+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:17.447 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:17 smithi122 systemd[1]: Stopping Ceph osd.6 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:12:17.659 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:17 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:17.405+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:17.659 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:17 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:17.405+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:17.713 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:17.448+0000 7fc6bde74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:12:17.713 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:17.448+0000 7fc6bde74700 -1 osd.6 82 *** Got signal Terminated *** 2024-05-03T18:12:17.713 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:17.448+0000 7fc6bde74700 -1 osd.6 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:12:18.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:17 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:17.659+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:18.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:17 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:17.659+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:18.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:17 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:17.659+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:18.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:17.807+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:18.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:17.807+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:18.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:17.807+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.699934+0000 front 2024-05-03T18:11:50.699847+0000 (oldest deadline 2024-05-03T18:12:15.399758+0000) 2024-05-03T18:12:18.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:17.782+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:18.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:17.782+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:18.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:17 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:17.782+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:18.293 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:18 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:18.112+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:18.293 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:18 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:18.112+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:18.294 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:17 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:17.954+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:18.294 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:17 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:17.954+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:18.294 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:17 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:17.954+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:18.540 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:18 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:18.393+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:18.540 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:18 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:18.393+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:19.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:18 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:18.638+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:19.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:18 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:18.638+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:19.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:18 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:18.638+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:19.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:18 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:18.760+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:19.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:18 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:18.760+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:19.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:18 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:18.760+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.699934+0000 front 2024-05-03T18:11:50.699847+0000 (oldest deadline 2024-05-03T18:12:15.399758+0000) 2024-05-03T18:12:19.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:18 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:18.734+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:19.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:18 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:18.734+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:19.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:18 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:18.734+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:19.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:19 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:19.069+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:19.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:19 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:19.069+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:19.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:18 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:18.985+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:19.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:18 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:18.985+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:19.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:18 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:18.985+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:19.790 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:19 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:19.430+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:19.790 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:19 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:19.430+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:19.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:19 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:19.661+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:19.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:19 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:19.661+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:19.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:19 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:19.661+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:19.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:19 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:19.661+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:20.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:19 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:19.773+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:20.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:19 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:19.773+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:20.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:19 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:19.773+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.699934+0000 front 2024-05-03T18:11:50.699847+0000 (oldest deadline 2024-05-03T18:12:15.399758+0000) 2024-05-03T18:12:20.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:19 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:19.771+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:20.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:19 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:19.771+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:20.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:19 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:19.771+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:20.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:20 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:20.052+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:20.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:20 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:20.052+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:20.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:19 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:19.948+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:20.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:19 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:19.948+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:20.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:19 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:19.948+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:20.702 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:20 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:20.404+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:20.702 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:20 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:20.404+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:21.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:20 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:20.703+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:21.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:20 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:20.703+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:21.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:20 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:20.703+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:21.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:20 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:20.703+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:21.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:20 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:20.766+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:21.214 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:20 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:20.766+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:21.214 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:20 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:20.766+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.699934+0000 front 2024-05-03T18:11:50.699847+0000 (oldest deadline 2024-05-03T18:12:15.399758+0000) 2024-05-03T18:12:21.214 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:20 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:20.766+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:58.201073+0000 front 2024-05-03T18:11:58.201107+0000 (oldest deadline 2024-05-03T18:12:20.501224+0000) 2024-05-03T18:12:21.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:20 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:20.758+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:21.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:20 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:20.758+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:21.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:20 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:20.758+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:21.215 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:20 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:20.758+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:21.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:20 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:20.962+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:21.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:20 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:20.962+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:21.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:20 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:20.962+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:21.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:21 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:21.027+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:21.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:21 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:21.027+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:21.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:21 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:21.027+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:21.737 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:21 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:21.445+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:21.738 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:21 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:21.445+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:21.738 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:21 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:21.445+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:57.111567+0000 front 2024-05-03T18:11:57.111734+0000 (oldest deadline 2024-05-03T18:12:21.211578+0000) 2024-05-03T18:12:22.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:21 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:21.739+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:22.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:21 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:21.739+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:22.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:21 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:21.739+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:22.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:21 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:21.739+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:22.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:21 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:21.805+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.196464+0000 front 2024-05-03T18:11:32.196646+0000 (oldest deadline 2024-05-03T18:11:58.096557+0000) 2024-05-03T18:12:22.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:21 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:21.805+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:40.898020+0000 front 2024-05-03T18:11:40.897960+0000 (oldest deadline 2024-05-03T18:12:04.998011+0000) 2024-05-03T18:12:22.214 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:21 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:21.805+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.699934+0000 front 2024-05-03T18:11:50.699847+0000 (oldest deadline 2024-05-03T18:12:15.399758+0000) 2024-05-03T18:12:22.214 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:21 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6[55342]: 2024-05-03T18:12:21.805+0000 7fc6b6460700 -1 osd.6 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:58.201073+0000 front 2024-05-03T18:11:58.201107+0000 (oldest deadline 2024-05-03T18:12:20.501224+0000) 2024-05-03T18:12:22.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:21 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:21.780+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:22.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:21 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:21.780+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:22.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:21 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:21.780+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:22.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:21 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:21.780+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:22.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:21 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:21.934+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:22.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:21 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:21.934+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:22.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:21 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:21.934+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:22.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:22 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:22.038+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:22.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:22 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:22.038+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:22.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:22 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:22.038+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:22.713 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:22 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:22.464+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:22.713 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:22 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:22.464+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:22.713 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:22 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:22.464+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:57.111567+0000 front 2024-05-03T18:11:57.111734+0000 (oldest deadline 2024-05-03T18:12:21.211578+0000) 2024-05-03T18:12:23.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:22 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:22.713+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:23.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:22 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:22.713+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:23.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:22 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:22.713+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:23.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:22 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:22.713+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:23.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:22 smithi122 podman[88827]: 2024-05-03 18:12:22.96728411 +0000 UTC m=+5.624729625 container died 8b4924e76ff8be6a60689fcc353cf067049fc9a883f4eedb0ab18f5de8553b7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-05-03T18:12:23.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:22 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:22.830+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:23.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:22 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:22.830+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:23.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:22 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:22.830+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:23.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:22 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:22.830+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:23.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:22 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:22.981+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:23.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:22 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:22.981+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:23.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:22 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:22.981+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:23.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:23 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:23.046+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:23.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:23 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:23.046+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:23.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:23 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:23.046+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:23.623 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:23 smithi122 podman[88827]: 2024-05-03 18:12:23.340976898 +0000 UTC m=+5.998422408 container cleanup 8b4924e76ff8be6a60689fcc353cf067049fc9a883f4eedb0ab18f5de8553b7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:12:23.623 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:23 smithi122 bash[88827]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6 2024-05-03T18:12:23.713 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:23 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:23.429+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:23.713 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:23 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:23.429+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:23.713 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:23 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:23.429+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:57.111567+0000 front 2024-05-03T18:11:57.111734+0000 (oldest deadline 2024-05-03T18:12:21.211578+0000) 2024-05-03T18:12:23.941 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:23 smithi122 podman[88857]: 2024-05-03 18:12:23.625838814 +0000 UTC m=+0.651506438 container remove 8b4924e76ff8be6a60689fcc353cf067049fc9a883f4eedb0ab18f5de8553b7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:12:23.941 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:23 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:23.863+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:23.941 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:23 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:23.863+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:23.942 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:23 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:23.863+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:23.942 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:23 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:23.863+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:24.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:23 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:23.714+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:24.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:23 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:23.714+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:24.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:23 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:23.714+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:24.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:23 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:23.714+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:24.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:23 smithi122 podman[88951]: 2024-05-03 18:12:23.943491674 +0000 UTC m=+0.100690441 container create 5594cbd4df803b46f1a647e1cc9ee2a64eef9273b63d1f6c960f569eb3c5ff4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:12:24.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:23 smithi122 podman[88951]: 2024-05-03 18:12:23.861473966 +0000 UTC m=+0.018672749 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:12:24.213 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:24 smithi122 podman[88951]: 2024-05-03 18:12:24.179500334 +0000 UTC m=+0.336699102 container init 5594cbd4df803b46f1a647e1cc9ee2a64eef9273b63d1f6c960f569eb3c5ff4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-03T18:12:24.214 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:24 smithi122 podman[88951]: 2024-05-03 18:12:24.182638913 +0000 UTC m=+0.339837689 container start 5594cbd4df803b46f1a647e1cc9ee2a64eef9273b63d1f6c960f569eb3c5ff4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:12:24.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:24 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:24.001+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:24.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:24 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:24.001+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:24.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:24 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:24.001+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:24.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:24 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:24.022+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:24.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:24 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:24.022+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:24.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:24 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:24.022+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:24.473 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:24 smithi122 podman[88951]: 2024-05-03 18:12:24.254617741 +0000 UTC m=+0.411816521 container attach 5594cbd4df803b46f1a647e1cc9ee2a64eef9273b63d1f6c960f569eb3c5ff4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:12:24.679 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:24 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:24.412+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:24.679 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:24 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:24.412+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:24.679 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:24 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:24.412+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:57.111567+0000 front 2024-05-03T18:11:57.111734+0000 (oldest deadline 2024-05-03T18:12:21.211578+0000) 2024-05-03T18:12:24.806 INFO:journalctl@ceph.osd.6.smithi122.stdout:May 03 18:12:24 smithi122 podman[88951]: 2024-05-03 18:12:24.476362796 +0000 UTC m=+0.633561577 container died 5594cbd4df803b46f1a647e1cc9ee2a64eef9273b63d1f6c960f569eb3c5ff4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-03T18:12:25.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:24 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:24.680+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:25.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:24 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:24.680+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:25.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:24 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:24.680+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:25.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:24 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:24.680+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:25.099 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:24 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:24.844+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:25.099 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:24 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:24.844+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:25.099 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:24 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:24.844+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:25.099 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:24 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:24.844+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:25.114 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.6.service' 2024-05-03T18:12:25.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:12:25.193 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-03T18:12:25.194 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-03T18:12:25.194 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.7 2024-05-03T18:12:25.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:25 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:25.027+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:25.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:25 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:25.027+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:25.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:25 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:25.027+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:25.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:24 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:24.996+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:25.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:24 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:24.996+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:25.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:24 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:24.996+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:25.487 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:25 smithi149 systemd[1]: Stopping Ceph osd.7 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:12:25.488 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:25 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:25.399+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:25.488 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:25 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:25.399+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:25.488 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:25 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:25.399+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:57.111567+0000 front 2024-05-03T18:11:57.111734+0000 (oldest deadline 2024-05-03T18:12:21.211578+0000) 2024-05-03T18:12:25.790 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:25 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:25.488+0000 7f3e454e6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:12:25.790 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:25 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:25.488+0000 7f3e454e6700 -1 osd.7 82 *** Got signal Terminated *** 2024-05-03T18:12:25.791 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:25 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:25.488+0000 7f3e454e6700 -1 osd.7 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:12:25.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:25 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:25.678+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:25.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:25 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:25.678+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:25.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:25 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:25.678+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:25.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:25 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:25.678+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:26.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:25 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:25.867+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:26.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:25 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:25.867+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:26.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:25 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:25.867+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:26.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:25 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:25.867+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:26.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:25 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:25.953+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:26.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:25 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:25.953+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:26.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:25 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:25.953+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:26.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:25 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:25.987+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:26.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:25 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:25.987+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:26.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:25 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:25.987+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:26.790 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:26 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:26.428+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:26.791 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:26 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:26.428+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:26.791 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:26 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:26.428+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:57.111567+0000 front 2024-05-03T18:11:57.111734+0000 (oldest deadline 2024-05-03T18:12:21.211578+0000) 2024-05-03T18:12:26.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:26 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:26.638+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:26.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:26 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:26.638+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:26.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:26 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:26.638+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:26.791 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:26 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:26.638+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:27.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:26 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:26.827+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:27.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:26 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:26.827+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:27.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:26 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:26.827+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:27.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:26 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:26.827+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:27.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:26 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:26.991+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:27.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:26 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:26.991+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:27.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:26 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:26.991+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:27.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:26 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:26.945+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:27.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:26 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:26.945+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:27.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:26 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:26.945+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:27.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:27 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:27.457+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:27.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:27 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:27.457+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:27.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:27 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:27.457+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:57.111567+0000 front 2024-05-03T18:11:57.111734+0000 (oldest deadline 2024-05-03T18:12:21.211578+0000) 2024-05-03T18:12:27.751 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:27 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:27.613+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:27.751 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:27 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:27.613+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:27.751 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:27 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:27.613+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:27.751 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:27 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:27.613+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:28.119 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:28.007+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:28.119 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:28.007+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:28.119 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:28.007+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:28.120 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:27 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:27.995+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:28.120 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:27 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:27.995+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:28.120 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:27 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:27.995+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:28.120 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:27 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:27.995+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:02.347975+0000 front 2024-05-03T18:12:02.348042+0000 (oldest deadline 2024-05-03T18:12:27.647978+0000) 2024-05-03T18:12:28.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:27 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:27.858+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:28.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:27 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:27.858+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:28.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:27 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:27.858+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:28.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:27 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:27.858+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:28.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:28 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:28.451+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:28.645 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:28 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:28.451+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:28.645 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:28 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:28.451+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:57.111567+0000 front 2024-05-03T18:11:57.111734+0000 (oldest deadline 2024-05-03T18:12:21.211578+0000) 2024-05-03T18:12:28.645 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:28 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:28.451+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.513877+0000 front 2024-05-03T18:12:04.513893+0000 (oldest deadline 2024-05-03T18:12:28.014004+0000) 2024-05-03T18:12:29.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:28 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:28.646+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:29.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:28 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:28.646+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:29.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:28 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:28.646+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:29.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:28 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:28.646+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:29.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:28 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:28.646+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:29.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:28 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:28.890+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:29.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:28 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:28.890+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:29.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:28 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:28.890+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:29.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:28 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:28.890+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:29.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:28.973+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:29.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:28.973+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:29.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:28.973+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:29.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:28.973+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:29.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:28.952+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:29.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:28.952+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:29.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:28.952+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:29.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:28 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:28.952+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:02.347975+0000 front 2024-05-03T18:12:02.348042+0000 (oldest deadline 2024-05-03T18:12:27.647978+0000) 2024-05-03T18:12:29.674 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:29.425+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:29.674 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:29.425+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:29.675 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:29.425+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:57.111567+0000 front 2024-05-03T18:11:57.111734+0000 (oldest deadline 2024-05-03T18:12:21.211578+0000) 2024-05-03T18:12:29.675 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:29.425+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.513877+0000 front 2024-05-03T18:12:04.513893+0000 (oldest deadline 2024-05-03T18:12:28.014004+0000) 2024-05-03T18:12:30.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:29.676+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:30.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:29.676+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:30.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:29.676+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:30.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:29.676+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:30.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:29 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:29.676+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:30.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:29 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:29.882+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:30.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:29 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:29.882+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:30.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:29 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:29.882+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:30.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:29 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:29.882+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:30.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:29.966+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:30.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:29.966+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:30.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:29.966+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:30.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:29.966+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:30.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:29.948+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:30.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:29.948+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:30.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:29.948+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:30.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:29 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:29.948+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:02.347975+0000 front 2024-05-03T18:12:02.348042+0000 (oldest deadline 2024-05-03T18:12:27.647978+0000) 2024-05-03T18:12:30.790 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:30 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:30.452+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.608183+0000 front 2024-05-03T18:11:39.608388+0000 (oldest deadline 2024-05-03T18:12:03.108411+0000) 2024-05-03T18:12:30.791 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:30 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:30.452+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.910299+0000 front 2024-05-03T18:11:48.910174+0000 (oldest deadline 2024-05-03T18:12:14.810216+0000) 2024-05-03T18:12:30.791 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:30 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:30.452+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:57.111567+0000 front 2024-05-03T18:11:57.111734+0000 (oldest deadline 2024-05-03T18:12:21.211578+0000) 2024-05-03T18:12:30.791 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:30 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7[51495]: 2024-05-03T18:12:30.452+0000 7f3e3dad2700 -1 osd.7 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.513877+0000 front 2024-05-03T18:12:04.513893+0000 (oldest deadline 2024-05-03T18:12:28.014004+0000) 2024-05-03T18:12:30.792 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:30 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:30.670+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:30.792 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:30 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:30.670+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:30.792 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:30 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:30.670+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:30.793 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:30 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:30.670+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:30.793 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:30 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:30.670+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:31.040 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:30 smithi149 podman[68884]: 2024-05-03 18:12:30.7919268 +0000 UTC m=+5.396278755 container died 417c7c8d1e2ea3bb80a30153a555777541b000f781dfa7f7d80d3b50ce884be9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-03T18:12:31.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:30 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:30.890+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:31.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:30 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:30.890+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:31.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:30 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:30.890+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:31.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:30 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:30.890+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:31.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:30 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:30.981+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:31.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:30 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:30.981+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:31.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:30 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:30.981+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:31.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:30 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:30.981+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:31.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:30 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:30.914+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:31.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:30 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:30.914+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:31.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:30 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:30.914+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:31.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:30 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:30.914+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:02.347975+0000 front 2024-05-03T18:12:02.348042+0000 (oldest deadline 2024-05-03T18:12:27.647978+0000) 2024-05-03T18:12:31.405 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:31 smithi149 podman[68884]: 2024-05-03 18:12:31.172718271 +0000 UTC m=+5.777070219 container cleanup 417c7c8d1e2ea3bb80a30153a555777541b000f781dfa7f7d80d3b50ce884be9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:12:31.405 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:31 smithi149 bash[68884]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7 2024-05-03T18:12:31.700 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:31 smithi149 podman[68902]: 2024-05-03 18:12:31.407622444 +0000 UTC m=+0.609504404 container remove 417c7c8d1e2ea3bb80a30153a555777541b000f781dfa7f7d80d3b50ce884be9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-03T18:12:32.041 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:31 smithi149 podman[68997]: 2024-05-03 18:12:31.630779348 +0000 UTC m=+0.018324569 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:12:32.041 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:31 smithi149 podman[68997]: 2024-05-03 18:12:31.758724609 +0000 UTC m=+0.146269813 container create 8933ce2b75ffb0313ceeeeeaf0748fa3b4fd7c89920dea99b262a37817cf8968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-03T18:12:32.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:31 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:31.701+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:32.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:31 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:31.701+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:32.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:31 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:31.701+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:32.042 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:31 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:31.701+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:32.042 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:31 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:31.701+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:32.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:31 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:31.862+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:32.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:31 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:31.862+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:32.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:31 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:31.862+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:32.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:31 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:31.862+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:32.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:31 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:31.862+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:32.333 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:32 smithi149 podman[68997]: 2024-05-03 18:12:32.044241651 +0000 UTC m=+0.431786866 container init 8933ce2b75ffb0313ceeeeeaf0748fa3b4fd7c89920dea99b262a37817cf8968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-03T18:12:32.333 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:32 smithi149 podman[68997]: 2024-05-03 18:12:32.047298933 +0000 UTC m=+0.434844147 container start 8933ce2b75ffb0313ceeeeeaf0748fa3b4fd7c89920dea99b262a37817cf8968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:12:32.334 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:32 smithi149 podman[68997]: 2024-05-03 18:12:32.128246372 +0000 UTC m=+0.515791594 container attach 8933ce2b75ffb0313ceeeeeaf0748fa3b4fd7c89920dea99b262a37817cf8968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-03T18:12:32.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:31 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:31.971+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:32.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:31 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:31.971+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:32.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:31 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:31.971+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:32.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:31 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:31.971+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:32.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:31 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:31.892+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:32.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:31 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:31.892+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:32.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:31 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:31.892+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:32.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:31 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:31.892+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:02.347975+0000 front 2024-05-03T18:12:02.348042+0000 (oldest deadline 2024-05-03T18:12:27.647978+0000) 2024-05-03T18:12:32.612 INFO:journalctl@ceph.osd.7.smithi149.stdout:May 03 18:12:32 smithi149 podman[68997]: 2024-05-03 18:12:32.336144605 +0000 UTC m=+0.723689813 container died 8933ce2b75ffb0313ceeeeeaf0748fa3b4fd7c89920dea99b262a37817cf8968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240430) 2024-05-03T18:12:32.879 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:32 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:32.738+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:32.880 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:32 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:32.738+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:32.880 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:32 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:32.738+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:32.880 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:32 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:32.738+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:32.880 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:32 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:32.738+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:32.900 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.7.service' 2024-05-03T18:12:32.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:12:32.939 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-03T18:12:32.939 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-03T18:12:32.939 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.8 2024-05-03T18:12:33.119 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:32.934+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:33.119 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:32.934+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:33.119 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:32.934+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:33.120 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:32.934+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:33.120 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:32.853+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:33.120 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:32.853+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:33.120 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:32.853+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:33.120 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:32.853+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:02.347975+0000 front 2024-05-03T18:12:02.348042+0000 (oldest deadline 2024-05-03T18:12:27.647978+0000) 2024-05-03T18:12:33.120 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:32 smithi192 systemd[1]: Stopping Ceph osd.8 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:12:33.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:32 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:32.875+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:33.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:32 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:32.875+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:33.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:32 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:32.875+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:33.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:32 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:32.875+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:33.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:32 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:32.875+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:33.619 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:33.255+0000 7f2849027700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:12:33.619 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:33.255+0000 7f2849027700 -1 osd.8 82 *** Got signal Terminated *** 2024-05-03T18:12:33.619 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:33.255+0000 7f2849027700 -1 osd.8 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:12:34.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:33 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:33.786+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:34.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:33 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:33.786+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:34.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:33 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:33.786+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:34.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:33 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:33.786+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:34.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:33 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:33.786+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:34.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:33 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:33.854+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:34.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:33 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:33.854+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:34.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:33 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:33.854+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:34.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:33 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:33.854+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:34.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:33 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:33.854+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:34.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:33.978+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:34.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:33.978+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:34.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:33.978+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:34.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:33.978+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:34.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:33.882+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:34.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:33.882+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:34.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:33.882+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:34.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:33 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:33.882+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:02.347975+0000 front 2024-05-03T18:12:02.348042+0000 (oldest deadline 2024-05-03T18:12:27.647978+0000) 2024-05-03T18:12:35.184 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:34 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:34.826+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:35.185 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:34 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:34.826+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:35.185 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:34 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:34.826+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:35.185 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:34 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:34.826+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:35.185 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:34 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:34.826+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:35.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:34.821+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:35.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:34.821+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:35.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:34.821+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:35.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:34.821+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:35.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:34 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:34.821+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:35.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:34 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:34.965+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:35.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:34 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:34.965+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:35.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:34 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:34.965+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:35.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:34 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:34.965+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:35.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:34 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:34.911+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:35.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:34 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:34.911+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:35.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:34 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:34.911+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:35.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:34 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:34.911+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:02.347975+0000 front 2024-05-03T18:12:02.348042+0000 (oldest deadline 2024-05-03T18:12:27.647978+0000) 2024-05-03T18:12:36.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:35 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:35.793+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:36.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:35 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:35.793+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:36.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:35 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:35.793+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:36.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:35 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:35.793+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:36.215 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:35 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:35.793+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:36.290 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:35 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:35.807+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:36.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:35 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:35.807+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:36.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:35 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:35.807+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:36.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:35 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:35.807+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:36.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:35 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:35.807+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:36.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:35 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:35.977+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:36.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:35 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:35.977+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:36.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:35 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:35.977+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:36.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:35 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:35.977+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:36.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:35 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:35.959+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:36.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:35 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:35.959+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:36.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:35 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:35.959+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:36.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:35 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:35.959+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:02.347975+0000 front 2024-05-03T18:12:02.348042+0000 (oldest deadline 2024-05-03T18:12:27.647978+0000) 2024-05-03T18:12:36.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:35 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:35.959+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:12.749679+0000 front 2024-05-03T18:12:12.749740+0000 (oldest deadline 2024-05-03T18:12:35.049825+0000) 2024-05-03T18:12:37.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:36 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:36.840+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:37.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:36 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:36.840+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:37.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:36 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:36.840+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:37.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:36 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:36.840+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:37.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:36 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:36.840+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:37.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:36 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:36.837+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:37.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:36 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:36.837+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:37.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:36 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:36.837+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:37.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:36 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:36.837+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:37.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:36 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:36.837+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:37.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:37 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:37.025+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:37.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:37 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:37.025+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:37.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:37 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:37.025+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:37.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:37 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:37.025+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:37.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:36 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:36.971+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:37.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:36 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:36.971+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:37.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:36 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:36.971+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:37.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:36 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:36.971+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:02.347975+0000 front 2024-05-03T18:12:02.348042+0000 (oldest deadline 2024-05-03T18:12:27.647978+0000) 2024-05-03T18:12:37.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:36 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:36.971+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:12.749679+0000 front 2024-05-03T18:12:12.749740+0000 (oldest deadline 2024-05-03T18:12:35.049825+0000) 2024-05-03T18:12:38.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:37 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:37.855+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:38.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:37 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:37.855+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:38.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:37 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:37.855+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:38.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:37 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:37.855+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:38.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:37 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:37.855+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:38.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:37 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:37.855+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:11.665756+0000 front 2024-05-03T18:12:11.665891+0000 (oldest deadline 2024-05-03T18:12:37.565592+0000) 2024-05-03T18:12:38.290 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:37 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:37.872+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:38.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:37 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:37.872+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:38.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:37 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:37.872+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:38.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:37 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:37.872+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:38.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:37 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:37.872+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:38.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:38.025+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:38.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:38.025+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:38.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:38.025+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:38.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:38.025+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:38.370 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:38.008+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.443150+0000 front 2024-05-03T18:11:31.443186+0000 (oldest deadline 2024-05-03T18:11:55.543125+0000) 2024-05-03T18:12:38.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:38.008+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:50.245336+0000 front 2024-05-03T18:11:50.245462+0000 (oldest deadline 2024-05-03T18:12:13.046044+0000) 2024-05-03T18:12:38.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:38.008+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.547106+0000 front 2024-05-03T18:11:56.547197+0000 (oldest deadline 2024-05-03T18:12:20.647001+0000) 2024-05-03T18:12:38.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:38.008+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:02.347975+0000 front 2024-05-03T18:12:02.348042+0000 (oldest deadline 2024-05-03T18:12:27.647978+0000) 2024-05-03T18:12:38.371 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8[52271]: 2024-05-03T18:12:38.008+0000 7f2841613700 -1 osd.8 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:12.749679+0000 front 2024-05-03T18:12:12.749740+0000 (oldest deadline 2024-05-03T18:12:35.049825+0000) 2024-05-03T18:12:39.030 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:38 smithi192 podman[69935]: 2024-05-03 18:12:38.739641773 +0000 UTC m=+5.600933387 container died 76ca6f555f23fa10318c6798ddf457abc3b4a6feff0df157612e7fc20520fe6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-03T18:12:39.030 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:38.984+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:39.031 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:38.984+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:39.031 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:38.984+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:39.031 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:38.984+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:39.031 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:38 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:38.984+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:39.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:38 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:38.893+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:39.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:38 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:38.893+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:39.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:38 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:38.893+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:39.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:38 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:38.893+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:39.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:38 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:38.893+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:39.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:38 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:38.893+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:11.665756+0000 front 2024-05-03T18:12:11.665891+0000 (oldest deadline 2024-05-03T18:12:37.565592+0000) 2024-05-03T18:12:39.290 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:38 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:38.828+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:39.292 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:38 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:38.828+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:39.292 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:38 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:38.828+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:39.292 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:38 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:38.828+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:39.292 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:38 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:38.828+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:39.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:39 smithi192 podman[69935]: 2024-05-03 18:12:39.131093243 +0000 UTC m=+5.992384852 container cleanup 76ca6f555f23fa10318c6798ddf457abc3b4a6feff0df157612e7fc20520fe6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-03T18:12:39.369 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:39 smithi192 bash[69935]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8 2024-05-03T18:12:39.743 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:39 smithi192 podman[69952]: 2024-05-03 18:12:39.407297246 +0000 UTC m=+0.664922037 container remove 76ca6f555f23fa10318c6798ddf457abc3b4a6feff0df157612e7fc20520fe6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-03T18:12:39.996 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:39 smithi192 podman[70046]: 2024-05-03 18:12:39.648216559 +0000 UTC m=+0.018335550 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:12:39.996 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:39 smithi192 podman[70046]: 2024-05-03 18:12:39.766550636 +0000 UTC m=+0.136669668 container create cdf4f3333994ff0f1d25361eef6c14a9c425a29acc1478bac09e409ebe96e3d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-03T18:12:39.996 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:39 smithi192 podman[70046]: 2024-05-03 18:12:39.994997652 +0000 UTC m=+0.365116635 container init cdf4f3333994ff0f1d25361eef6c14a9c425a29acc1478bac09e409ebe96e3d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-03T18:12:39.997 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:39 smithi192 podman[70046]: 2024-05-03 18:12:39.998430738 +0000 UTC m=+0.368549719 container start cdf4f3333994ff0f1d25361eef6c14a9c425a29acc1478bac09e409ebe96e3d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-03T18:12:40.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:39 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:39.783+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:40.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:39 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:39.783+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:40.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:39 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:39.783+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:40.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:39 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:39.783+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:40.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:39 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:39.783+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:40.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:39 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:39.783+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:40.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:39 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:39.925+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:40.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:39 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:39.925+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:40.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:39 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:39.925+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:40.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:39 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:39.925+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:40.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:39 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:39.925+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:40.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:39 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:39.925+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:11.665756+0000 front 2024-05-03T18:12:11.665891+0000 (oldest deadline 2024-05-03T18:12:37.565592+0000) 2024-05-03T18:12:40.286 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:40 smithi192 podman[70046]: 2024-05-03 18:12:40.110582861 +0000 UTC m=+0.480701869 container attach cdf4f3333994ff0f1d25361eef6c14a9c425a29acc1478bac09e409ebe96e3d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux ) 2024-05-03T18:12:40.287 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:40 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:40.022+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:40.287 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:40 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:40.022+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:40.287 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:40 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:40.022+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:40.287 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:40 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:40.022+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:40.287 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:40 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:40.022+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:40.559 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 03 18:12:40 smithi192 podman[70046]: 2024-05-03 18:12:40.289029271 +0000 UTC m=+0.659148260 container died cdf4f3333994ff0f1d25361eef6c14a9c425a29acc1478bac09e409ebe96e3d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-03T18:12:40.914 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.8.service' 2024-05-03T18:12:40.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:12:40.951 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-03T18:12:40.952 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-03T18:12:40.952 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.9 2024-05-03T18:12:41.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:40 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:40.924+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:41.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:40 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:40.924+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:41.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:40 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:40.924+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:41.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:40 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:40.924+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:41.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:40 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:40.924+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:41.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:40 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:40.924+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:11.665756+0000 front 2024-05-03T18:12:11.665891+0000 (oldest deadline 2024-05-03T18:12:37.565592+0000) 2024-05-03T18:12:41.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:41 smithi122 systemd[1]: Stopping Ceph osd.9 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:12:41.290 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:40 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:40.821+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:41.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:40 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:40.821+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:41.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:40 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:40.821+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:41.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:40 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:40.821+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:41.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:40 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:40.821+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:41.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:40 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:40.821+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:41.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:41 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:41.030+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:41.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:41 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:41.030+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:41.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:41 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:41.030+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:41.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:41 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:41.030+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:41.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:41 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:41.030+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:41.713 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:41 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:41.244+0000 7f0cfe4b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:12:41.714 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:41 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:41.244+0000 7f0cfe4b1700 -1 osd.9 82 *** Got signal Terminated *** 2024-05-03T18:12:41.714 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:41 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:41.244+0000 7f0cfe4b1700 -1 osd.9 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:12:42.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:41 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:41.891+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:42.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:41 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:41.891+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:42.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:41 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:41.891+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:42.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:41 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:41.891+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:42.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:41 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:41.891+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:42.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:41 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:41.891+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:11.665756+0000 front 2024-05-03T18:12:11.665891+0000 (oldest deadline 2024-05-03T18:12:37.565592+0000) 2024-05-03T18:12:42.290 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:41 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:41.822+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:42.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:41 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:41.822+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:42.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:41 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:41.822+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:42.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:41 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:41.822+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:42.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:41 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:41.822+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:42.292 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:41 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:41.822+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:42.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:42 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:42.027+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:42.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:42 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:42.027+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:42.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:42 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:42.027+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:42.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:42 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:42.027+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:42.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:42 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:42.027+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:43.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:42 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:42.791+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:43.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:42 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:42.791+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:43.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:42 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:42.791+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:43.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:42 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:42.791+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:43.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:42 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:42.791+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:43.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:42 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:42.791+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:43.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:42 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:42.854+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:43.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:42 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:42.854+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:43.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:42 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:42.854+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:43.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:42 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:42.854+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:43.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:42 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:42.854+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:43.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:42 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:42.854+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:11.665756+0000 front 2024-05-03T18:12:11.665891+0000 (oldest deadline 2024-05-03T18:12:37.565592+0000) 2024-05-03T18:12:43.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:43 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:43.017+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:43.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:43 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:43.017+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:43.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:43 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:43.017+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:43.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:43 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:43.017+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:43.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:43 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:43.017+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:44.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:43 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:43.779+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:44.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:43 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:43.779+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:44.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:43 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:43.779+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:44.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:43 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:43.779+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:44.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:43 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:43.779+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:44.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:43 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:43.779+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:44.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:43 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:43.848+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:44.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:43 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:43.848+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:44.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:43 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:43.848+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:44.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:43 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:43.848+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:44.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:43 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:43.848+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:44.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:43 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:43.848+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:11.665756+0000 front 2024-05-03T18:12:11.665891+0000 (oldest deadline 2024-05-03T18:12:37.565592+0000) 2024-05-03T18:12:44.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:43 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:43.992+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:44.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:43 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:43.992+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:44.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:43 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:43.992+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:44.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:43 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:43.992+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:44.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:43 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:43.992+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:45.040 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:44 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:44.817+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:45.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:44 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:44.817+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:45.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:44 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:44.817+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:45.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:44 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:44.817+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:45.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:44 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:44.817+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:45.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:44 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:44.817+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:45.042 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:44 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:44.817+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:22.011084+0000 front 2024-05-03T18:12:22.011190+0000 (oldest deadline 2024-05-03T18:12:44.310887+0000) 2024-05-03T18:12:45.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:44 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:44.844+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:45.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:44 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:44.844+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:45.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:44 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:44.844+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:45.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:44 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:44.844+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:45.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:44 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:44.844+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:45.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:44 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:44.844+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:11.665756+0000 front 2024-05-03T18:12:11.665891+0000 (oldest deadline 2024-05-03T18:12:37.565592+0000) 2024-05-03T18:12:45.293 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:45 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:45.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:45.293 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:45 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:45.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:45.294 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:45 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:45.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:45.294 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:45 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:45.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:45.294 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:45 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:45.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:46.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:45.836+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.360243+0000 front 2024-05-03T18:11:31.360139+0000 (oldest deadline 2024-05-03T18:11:55.460353+0000) 2024-05-03T18:12:46.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:45.836+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:39.561534+0000 front 2024-05-03T18:11:39.561407+0000 (oldest deadline 2024-05-03T18:12:04.261421+0000) 2024-05-03T18:12:46.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:45.836+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.362565+0000 front 2024-05-03T18:11:48.362755+0000 (oldest deadline 2024-05-03T18:12:13.062504+0000) 2024-05-03T18:12:46.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:45.836+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:54.163176+0000 front 2024-05-03T18:11:54.163222+0000 (oldest deadline 2024-05-03T18:12:20.063264+0000) 2024-05-03T18:12:46.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:45.836+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:06.365236+0000 front 2024-05-03T18:12:06.365220+0000 (oldest deadline 2024-05-03T18:12:31.665153+0000) 2024-05-03T18:12:46.214 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:45 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9[62013]: 2024-05-03T18:12:45.836+0000 7f0cf6a9d700 -1 osd.9 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:11.665756+0000 front 2024-05-03T18:12:11.665891+0000 (oldest deadline 2024-05-03T18:12:37.565592+0000) 2024-05-03T18:12:46.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:45 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:45.801+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:46.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:45 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:45.801+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:46.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:45 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:45.801+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:46.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:45 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:45.801+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:46.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:45 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:45.801+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:46.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:45 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:45.801+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:46.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:45 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:45.801+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:22.011084+0000 front 2024-05-03T18:12:22.011190+0000 (oldest deadline 2024-05-03T18:12:44.310887+0000) 2024-05-03T18:12:46.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:46 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:46.077+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:46.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:46 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:46.077+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:46.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:46 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:46.077+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:46.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:46 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:46.077+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:46.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:46 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:46.077+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:46.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:46 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:46.077+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:46.911 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:46 smithi122 podman[89055]: 2024-05-03 18:12:46.648589825 +0000 UTC m=+5.499991631 container died be4386647e23c40aea3524a9f03bc3a090f651a597f131c05962cf73c082fd8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-03T18:12:47.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:46 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:46.763+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:47.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:46 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:46.763+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:47.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:46 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:46.763+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:47.042 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:46 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:46.763+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:47.042 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:46 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:46.763+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:47.042 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:46 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:46.763+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:47.042 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:46 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:46.763+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:22.011084+0000 front 2024-05-03T18:12:22.011190+0000 (oldest deadline 2024-05-03T18:12:44.310887+0000) 2024-05-03T18:12:47.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:47 smithi122 podman[89055]: 2024-05-03 18:12:47.003421339 +0000 UTC m=+5.854823143 container cleanup be4386647e23c40aea3524a9f03bc3a090f651a597f131c05962cf73c082fd8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-05-03T18:12:47.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:47 smithi122 bash[89055]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9 2024-05-03T18:12:47.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:47 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:47.085+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:47.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:47 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:47.085+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:47.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:47 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:47.085+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:47.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:47 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:47.085+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:47.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:47 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:47.085+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:47.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:47 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:47.085+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:47.568 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:47 smithi122 podman[89072]: 2024-05-03 18:12:47.246438791 +0000 UTC m=+0.596522809 container remove be4386647e23c40aea3524a9f03bc3a090f651a597f131c05962cf73c082fd8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-05-03T18:12:47.865 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:47 smithi122 podman[89165]: 2024-05-03 18:12:47.473935567 +0000 UTC m=+0.018159248 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:12:47.865 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:47 smithi122 podman[89165]: 2024-05-03 18:12:47.57254986 +0000 UTC m=+0.116773599 container create 3c1cce2e0db8d45967ce79708c72b546fd5c7fbade424800b8310e818966e768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:12:47.865 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:47 smithi122 podman[89165]: 2024-05-03 18:12:47.792218292 +0000 UTC m=+0.336441972 container init 3c1cce2e0db8d45967ce79708c72b546fd5c7fbade424800b8310e818966e768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-03T18:12:47.865 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:47 smithi122 podman[89165]: 2024-05-03 18:12:47.795440752 +0000 UTC m=+0.339664432 container start 3c1cce2e0db8d45967ce79708c72b546fd5c7fbade424800b8310e818966e768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:12:47.865 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:47 smithi122 podman[89165]: 2024-05-03 18:12:47.867086777 +0000 UTC m=+0.411310468 container attach 3c1cce2e0db8d45967ce79708c72b546fd5c7fbade424800b8310e818966e768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9-deactivate, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430) 2024-05-03T18:12:48.213 INFO:journalctl@ceph.osd.9.smithi122.stdout:May 03 18:12:48 smithi122 podman[89165]: 2024-05-03 18:12:48.085034138 +0000 UTC m=+0.629257812 container died 3c1cce2e0db8d45967ce79708c72b546fd5c7fbade424800b8310e818966e768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-03T18:12:48.290 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:47 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:47.803+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:48.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:47 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:47.803+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:48.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:47 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:47.803+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:48.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:47 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:47.803+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:48.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:47 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:47.803+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:48.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:47 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:47.803+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:48.292 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:47 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:47.803+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:22.011084+0000 front 2024-05-03T18:12:22.011190+0000 (oldest deadline 2024-05-03T18:12:44.310887+0000) 2024-05-03T18:12:48.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:48 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:48.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:48.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:48 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:48.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:48.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:48 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:48.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:48.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:48 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:48.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:48.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:48 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:48.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:48.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:48 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:48.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:48.641 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.9.service' 2024-05-03T18:12:48.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:12:48.678 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-03T18:12:48.679 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-03T18:12:48.679 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.10 2024-05-03T18:12:48.999 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:48 smithi149 systemd[1]: Stopping Ceph osd.10 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:12:49.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:48 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:48.824+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:49.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:48 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:48.824+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:49.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:48 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:48.824+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:49.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:48 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:48.824+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:49.001 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:48 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:48.824+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:49.001 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:48 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:48.824+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:49.001 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:48 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:48.824+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:22.011084+0000 front 2024-05-03T18:12:22.011190+0000 (oldest deadline 2024-05-03T18:12:44.310887+0000) 2024-05-03T18:12:49.001 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:49 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:49.000+0000 7f3efba37700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:12:49.002 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:49 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:49.000+0000 7f3efba37700 -1 osd.10 82 *** Got signal Terminated *** 2024-05-03T18:12:49.002 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:49 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:49.000+0000 7f3efba37700 -1 osd.10 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:12:49.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:49 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:49.068+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:49.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:49 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:49.068+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:49.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:49 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:49.068+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:49.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:49 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:49.068+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:49.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:49 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:49.068+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:49.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:49 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:49.068+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:50.290 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:49 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:49.840+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:50.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:49 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:49.840+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:50.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:49 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:49.840+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:50.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:49 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:49.840+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:50.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:49 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:49.840+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:50.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:49 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:49.840+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:50.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:49 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:49.840+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:22.011084+0000 front 2024-05-03T18:12:22.011190+0000 (oldest deadline 2024-05-03T18:12:44.310887+0000) 2024-05-03T18:12:50.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:50 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:50.102+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:50.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:50 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:50.102+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:50.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:50 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:50.102+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:50.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:50 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:50.102+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:50.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:50 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:50.102+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:50.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:50 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:50.102+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:51.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:50 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:50.815+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:51.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:50 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:50.815+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:51.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:50 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:50.815+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:51.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:50 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:50.815+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:51.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:50 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:50.815+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:51.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:50 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:50.815+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:51.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:50 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:50.815+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:22.011084+0000 front 2024-05-03T18:12:22.011190+0000 (oldest deadline 2024-05-03T18:12:44.310887+0000) 2024-05-03T18:12:51.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:51 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:51.112+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:51.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:51 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:51.112+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:51.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:51 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:51.112+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:51.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:51 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:51.112+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:51.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:51 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:51.112+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:51.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:51 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:51.112+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:52.290 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:51 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:51.831+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:52.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:51 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:51.831+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:52.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:51 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:51.831+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:52.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:51 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:51.831+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:52.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:51 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:51.831+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:52.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:51 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:51.831+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:52.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:51 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:51.831+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:22.011084+0000 front 2024-05-03T18:12:22.011190+0000 (oldest deadline 2024-05-03T18:12:44.310887+0000) 2024-05-03T18:12:52.619 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:52 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:52.137+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:52.620 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:52 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:52.137+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:52.620 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:52 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:52.137+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:52.620 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:52 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:52.137+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:52.620 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:52 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:52.137+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:52.620 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:52 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:52.137+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:52.620 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:52 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:52.137+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-05-03T18:12:25.664379+0000 front 2024-05-03T18:12:25.664426+0000 (oldest deadline 2024-05-03T18:12:51.564394+0000) 2024-05-03T18:12:53.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:52 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:52.852+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:53.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:52 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:52.852+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:53.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:52 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:52.852+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:53.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:52 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:52.852+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:53.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:52 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:52.852+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:53.292 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:52 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:52.852+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:53.292 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:52 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:52.852+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:22.011084+0000 front 2024-05-03T18:12:22.011190+0000 (oldest deadline 2024-05-03T18:12:44.310887+0000) 2024-05-03T18:12:53.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:53 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:53.119+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:53.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:53 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:53.119+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:53.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:53 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:53.119+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:53.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:53 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:53.119+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:53.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:53 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:53.119+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:53.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:53 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:53.119+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:53.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:53 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:53.119+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-05-03T18:12:25.664379+0000 front 2024-05-03T18:12:25.664426+0000 (oldest deadline 2024-05-03T18:12:51.564394+0000) 2024-05-03T18:12:54.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:53 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:53.809+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:31.103843+0000 front 2024-05-03T18:11:31.104040+0000 (oldest deadline 2024-05-03T18:11:55.803875+0000) 2024-05-03T18:12:54.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:53 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:53.809+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:38.705010+0000 front 2024-05-03T18:11:38.704990+0000 (oldest deadline 2024-05-03T18:12:08.105604+0000) 2024-05-03T18:12:54.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:53 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:53.809+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:48.106409+0000 front 2024-05-03T18:11:48.106212+0000 (oldest deadline 2024-05-03T18:12:11.606123+0000) 2024-05-03T18:12:54.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:53 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:53.809+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-05-03T18:11:56.907313+0000 front 2024-05-03T18:11:56.907179+0000 (oldest deadline 2024-05-03T18:12:19.207239+0000) 2024-05-03T18:12:54.291 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:53 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:53.809+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.408560+0000 front 2024-05-03T18:12:04.408541+0000 (oldest deadline 2024-05-03T18:12:27.908627+0000) 2024-05-03T18:12:54.292 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:53 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:53.809+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.809962+0000 front 2024-05-03T18:12:13.810099+0000 (oldest deadline 2024-05-03T18:12:39.709947+0000) 2024-05-03T18:12:54.292 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:53 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10[58281]: 2024-05-03T18:12:53.809+0000 7f3ef802b700 -1 osd.10 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:22.011084+0000 front 2024-05-03T18:12:22.011190+0000 (oldest deadline 2024-05-03T18:12:44.310887+0000) 2024-05-03T18:12:54.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:54 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:54.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:54.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:54 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:54.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:54.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:54 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:54.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:54.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:54 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:54.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:54.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:54 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:54.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:54.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:54 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:54.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:54.371 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:54 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:54.074+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-05-03T18:12:25.664379+0000 front 2024-05-03T18:12:25.664426+0000 (oldest deadline 2024-05-03T18:12:51.564394+0000) 2024-05-03T18:12:54.582 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:54 smithi149 podman[69127]: 2024-05-03 18:12:54.313740554 +0000 UTC m=+5.439975490 container died 5e10b994845a8466d31c75dfa047fd31df5fdf549f5c1ae634f030ea6b7e091d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:12:54.908 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:54 smithi149 podman[69127]: 2024-05-03 18:12:54.675539539 +0000 UTC m=+5.801774472 container cleanup 5e10b994845a8466d31c75dfa047fd31df5fdf549f5c1ae634f030ea6b7e091d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:12:54.908 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:54 smithi149 bash[69127]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10 2024-05-03T18:12:55.225 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:54 smithi149 podman[69144]: 2024-05-03 18:12:54.909916391 +0000 UTC m=+0.598479736 container remove 5e10b994845a8466d31c75dfa047fd31df5fdf549f5c1ae634f030ea6b7e091d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-03T18:12:55.334 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:55 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:55.044+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:55.334 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:55 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:55.044+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:55.334 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:55 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:55.044+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:55.335 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:55 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:55.044+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:55.335 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:55 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:55.044+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:55.335 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:55 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:55.044+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:55.335 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:55 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:55.044+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-05-03T18:12:25.664379+0000 front 2024-05-03T18:12:25.664426+0000 (oldest deadline 2024-05-03T18:12:51.564394+0000) 2024-05-03T18:12:55.540 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:55 smithi149 podman[69232]: 2024-05-03 18:12:55.227464511 +0000 UTC m=+0.105211369 container create 856825d00c9f42750e8bee60dc13b0b5723ee94913dad3258b7397d989155eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:12:55.541 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:55 smithi149 podman[69232]: 2024-05-03 18:12:55.140426409 +0000 UTC m=+0.018173283 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:12:55.541 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:55 smithi149 podman[69232]: 2024-05-03 18:12:55.454782676 +0000 UTC m=+0.332529539 container init 856825d00c9f42750e8bee60dc13b0b5723ee94913dad3258b7397d989155eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-03T18:12:55.541 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:55 smithi149 podman[69232]: 2024-05-03 18:12:55.458062251 +0000 UTC m=+0.335809112 container start 856825d00c9f42750e8bee60dc13b0b5723ee94913dad3258b7397d989155eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5) 2024-05-03T18:12:56.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:55 smithi149 podman[69232]: 2024-05-03 18:12:55.564814047 +0000 UTC m=+0.442560924 container attach 856825d00c9f42750e8bee60dc13b0b5723ee94913dad3258b7397d989155eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-05-03T18:12:56.041 INFO:journalctl@ceph.osd.10.smithi149.stdout:May 03 18:12:55 smithi149 podman[69232]: 2024-05-03 18:12:55.751147115 +0000 UTC m=+0.628893991 container died 856825d00c9f42750e8bee60dc13b0b5723ee94913dad3258b7397d989155eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-05-03T18:12:56.358 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.10.service' 2024-05-03T18:12:56.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:56.092+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:56.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:56.092+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:56.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:56.092+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:56.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:56.092+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:56.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:56.092+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:56.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:56.092+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:56.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:56.092+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-05-03T18:12:25.664379+0000 front 2024-05-03T18:12:25.664426+0000 (oldest deadline 2024-05-03T18:12:51.564394+0000) 2024-05-03T18:12:56.397 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:12:56.397 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-03T18:12:56.397 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-03T18:12:56.398 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.11 2024-05-03T18:12:56.689 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:56 smithi192 systemd[1]: Stopping Ceph osd.11 for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:12:57.052 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:56.690+0000 7f90a5f91700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:12:57.052 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:56.690+0000 7f90a5f91700 -1 osd.11 82 *** Got signal Terminated *** 2024-05-03T18:12:57.052 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:56 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:56.690+0000 7f90a5f91700 -1 osd.11 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:12:57.053 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:57 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:57.053+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:57.053 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:57 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:57.053+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:57.053 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:57 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:57.053+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:57.053 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:57 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:57.053+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:57.053 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:57 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:57.053+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:57.053 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:57 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:57.053+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:57.053 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:57 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:57.053+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-05-03T18:12:25.664379+0000 front 2024-05-03T18:12:25.664426+0000 (oldest deadline 2024-05-03T18:12:51.564394+0000) 2024-05-03T18:12:58.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:58 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:58.018+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:58.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:58 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:58.018+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:58.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:58 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:58.018+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:58.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:58 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:58.018+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:58.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:58 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:58.018+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:58.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:58 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:58.018+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:58.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:58 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:58.018+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-05-03T18:12:25.664379+0000 front 2024-05-03T18:12:25.664426+0000 (oldest deadline 2024-05-03T18:12:51.564394+0000) 2024-05-03T18:12:59.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:59 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:59.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:12:59.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:59 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:59.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:12:59.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:59 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:59.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:12:59.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:59 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:59.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:12:59.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:59 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:59.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:12:59.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:59 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:59.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:12:59.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:12:59 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:12:59.035+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-05-03T18:12:25.664379+0000 front 2024-05-03T18:12:25.664426+0000 (oldest deadline 2024-05-03T18:12:51.564394+0000) 2024-05-03T18:13:00.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:00 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:00.034+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:13:00.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:00 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:00.034+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:13:00.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:00 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:00.034+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:13:00.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:00 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:00.034+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:13:00.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:00 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:00.034+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:13:00.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:00 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:00.034+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:13:00.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:00 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:00.034+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-05-03T18:12:25.664379+0000 front 2024-05-03T18:12:25.664426+0000 (oldest deadline 2024-05-03T18:12:51.564394+0000) 2024-05-03T18:13:00.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:00 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:00.034+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-05-03T18:12:36.165844+0000 front 2024-05-03T18:12:36.165889+0000 (oldest deadline 2024-05-03T18:12:59.665890+0000) 2024-05-03T18:13:01.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:01 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:01.011+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-05-03T18:11:32.854573+0000 front 2024-05-03T18:11:32.854456+0000 (oldest deadline 2024-05-03T18:11:55.154575+0000) 2024-05-03T18:13:01.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:01 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:01.011+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-05-03T18:11:41.457150+0000 front 2024-05-03T18:11:41.457299+0000 (oldest deadline 2024-05-03T18:12:05.557012+0000) 2024-05-03T18:13:01.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:01 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:01.011+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-03T18:11:49.459089+0000 front 2024-05-03T18:11:49.458979+0000 (oldest deadline 2024-05-03T18:12:12.959148+0000) 2024-05-03T18:13:01.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:01 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:01.011+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-05-03T18:12:04.061137+0000 front 2024-05-03T18:12:04.061123+0000 (oldest deadline 2024-05-03T18:12:28.761016+0000) 2024-05-03T18:13:01.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:01 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:01.011+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-03T18:12:13.962775+0000 front 2024-05-03T18:12:13.963001+0000 (oldest deadline 2024-05-03T18:12:38.063000+0000) 2024-05-03T18:13:01.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:01 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:01.011+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-05-03T18:12:18.063376+0000 front 2024-05-03T18:12:18.063338+0000 (oldest deadline 2024-05-03T18:12:45.663875+0000) 2024-05-03T18:13:01.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:01 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:01.011+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-05-03T18:12:25.664379+0000 front 2024-05-03T18:12:25.664426+0000 (oldest deadline 2024-05-03T18:12:51.564394+0000) 2024-05-03T18:13:01.370 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:01 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11[58882]: 2024-05-03T18:13:01.011+0000 7f909e57d700 -1 osd.11 82 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-05-03T18:12:36.165844+0000 front 2024-05-03T18:12:36.165889+0000 (oldest deadline 2024-05-03T18:12:59.665890+0000) 2024-05-03T18:13:02.369 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:02 smithi192 podman[70164]: 2024-05-03 18:13:02.110699009 +0000 UTC m=+5.520419113 container died 99d2745b2cfce5c98754954f933cb99907387f0d690c3f8e392fb5b9e953c802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430) 2024-05-03T18:13:02.869 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:02 smithi192 podman[70164]: 2024-05-03 18:13:02.449139221 +0000 UTC m=+5.858859323 container cleanup 99d2745b2cfce5c98754954f933cb99907387f0d690c3f8e392fb5b9e953c802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:13:02.870 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:02 smithi192 bash[70164]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11 2024-05-03T18:13:02.870 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:02 smithi192 podman[70181]: 2024-05-03 18:13:02.641887773 +0000 UTC m=+0.533710487 container remove 99d2745b2cfce5c98754954f933cb99907387f0d690c3f8e392fb5b9e953c802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, ceph=True) 2024-05-03T18:13:03.327 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:03 smithi192 podman[70271]: 2024-05-03 18:13:02.965544332 +0000 UTC m=+0.017823347 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:13:03.328 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:03 smithi192 podman[70271]: 2024-05-03 18:13:03.067699139 +0000 UTC m=+0.119978145 container create ec48529e2e8153b711fd39f0c4420df345ad7dadbb8106db59f14e0d6183161d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-03T18:13:03.328 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:03 smithi192 podman[70271]: 2024-05-03 18:13:03.253823264 +0000 UTC m=+0.306102269 container init ec48529e2e8153b711fd39f0c4420df345ad7dadbb8106db59f14e0d6183161d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-03T18:13:03.328 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:03 smithi192 podman[70271]: 2024-05-03 18:13:03.256999836 +0000 UTC m=+0.309278844 container start ec48529e2e8153b711fd39f0c4420df345ad7dadbb8106db59f14e0d6183161d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-03T18:13:03.619 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:03 smithi192 podman[70271]: 2024-05-03 18:13:03.328586681 +0000 UTC m=+0.380865697 container attach ec48529e2e8153b711fd39f0c4420df345ad7dadbb8106db59f14e0d6183161d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-03T18:13:03.620 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:03 smithi192 podman[70271]: 2024-05-03 18:13:03.549749377 +0000 UTC m=+0.602028375 container died ec48529e2e8153b711fd39f0c4420df345ad7dadbb8106db59f14e0d6183161d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, ceph=True) 2024-05-03T18:13:04.088 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 03 18:13:04 smithi192 podman[70296]: 2024-05-03 18:13:04.073093149 +0000 UTC m=+0.513559301 container remove ec48529e2e8153b711fd39f0c4420df345ad7dadbb8106db59f14e0d6183161d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-03T18:13:04.092 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@osd.11.service' 2024-05-03T18:13:04.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:04.129 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-03T18:13:04.129 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-03T18:13:04.129 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.a 2024-05-03T18:13:04.438 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 18:13:04 smithi122 systemd[1]: Stopping Ceph mds.a for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:04.713 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 18:13:04 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-a[69241]: 2024-05-03T18:13:04.439+0000 7f7ace166700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:04.713 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 18:13:04 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-a[69241]: 2024-05-03T18:13:04.439+0000 7f7ace166700 -1 mds.a *** got signal Terminated *** 2024-05-03T18:13:06.413 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 18:13:06 smithi122 podman[89280]: 2024-05-03 18:13:06.064241334 +0000 UTC m=+1.736991785 container died 818cdb0c4598034d4b01f9ead224f8beaeee419bd00ba2f185df68a15eea2ce7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=) 2024-05-03T18:13:06.713 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 18:13:06 smithi122 podman[89280]: 2024-05-03 18:13:06.51460605 +0000 UTC m=+2.187356493 container cleanup 818cdb0c4598034d4b01f9ead224f8beaeee419bd00ba2f185df68a15eea2ce7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-05-03T18:13:06.714 INFO:journalctl@ceph.mds.a.smithi122.stdout:May 03 18:13:06 smithi122 bash[89280]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-a 2024-05-03T18:13:06.947 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.a.service' 2024-05-03T18:13:06.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:06.981 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-03T18:13:06.982 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-03T18:13:06.982 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.d 2024-05-03T18:13:07.464 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 18:13:07 smithi122 systemd[1]: Stopping Ceph mds.d for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:07.464 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 18:13:07 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-d[67958]: 2024-05-03T18:13:07.293+0000 7f5d7822d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:07.464 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 18:13:07 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-d[67958]: 2024-05-03T18:13:07.293+0000 7f5d7822d700 -1 mds.d *** got signal Terminated *** 2024-05-03T18:13:08.962 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 18:13:08 smithi122 podman[89400]: 2024-05-03 18:13:08.636330055 +0000 UTC m=+1.442559667 container died 53dcd52b8677bbe0d97a2b7996a4019fffbe6e64ae4fcf939877296836195fc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux ) 2024-05-03T18:13:09.213 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 18:13:09 smithi122 podman[89400]: 2024-05-03 18:13:09.060500257 +0000 UTC m=+1.866729868 container cleanup 53dcd52b8677bbe0d97a2b7996a4019fffbe6e64ae4fcf939877296836195fc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-d, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:13:09.213 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 18:13:09 smithi122 bash[89400]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-d 2024-05-03T18:13:09.480 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.d.service' 2024-05-03T18:13:09.501 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 18:13:09 smithi122 podman[89414]: 2024-05-03 18:13:09.253969421 +0000 UTC m=+0.612795385 container remove 53dcd52b8677bbe0d97a2b7996a4019fffbe6e64ae4fcf939877296836195fc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-03T18:13:09.502 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 18:13:09 smithi122 systemd[1]: ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.d.service: Deactivated successfully. 2024-05-03T18:13:09.502 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 18:13:09 smithi122 systemd[1]: Stopped Ceph mds.d for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T18:13:09.502 INFO:journalctl@ceph.mds.d.smithi122.stdout:May 03 18:13:09 smithi122 systemd[1]: ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.d.service: Consumed 1.141s CPU time. 2024-05-03T18:13:09.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:09.511 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-03T18:13:09.512 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-03T18:13:09.512 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.g 2024-05-03T18:13:09.821 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 18:13:09 smithi122 systemd[1]: Stopping Ceph mds.g for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:10.213 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 18:13:09 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-g[68796]: 2024-05-03T18:13:09.821+0000 7fc32f787700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:10.214 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 18:13:09 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-g[68796]: 2024-05-03T18:13:09.821+0000 7fc32f787700 -1 mds.g *** got signal Terminated *** 2024-05-03T18:13:13.947 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 18:13:13 smithi122 podman[89521]: 2024-05-03 18:13:13.602933084 +0000 UTC m=+3.868774231 container died c5e8610b8cf310ff5d326a4f754c0212eb6730753e20edc0bbbf73e1b2600391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-03T18:13:14.213 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 18:13:14 smithi122 podman[89521]: 2024-05-03 18:13:14.031614174 +0000 UTC m=+4.297455320 container cleanup c5e8610b8cf310ff5d326a4f754c0212eb6730753e20edc0bbbf73e1b2600391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:13:14.214 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 18:13:14 smithi122 bash[89521]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-g 2024-05-03T18:13:14.214 INFO:journalctl@ceph.mds.g.smithi122.stdout:May 03 18:13:14 smithi122 podman[89536]: 2024-05-03 18:13:14.199405474 +0000 UTC m=+0.590932245 container remove c5e8610b8cf310ff5d326a4f754c0212eb6730753e20edc0bbbf73e1b2600391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430) 2024-05-03T18:13:14.404 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.g.service' 2024-05-03T18:13:14.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:14.435 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-03T18:13:14.435 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-03T18:13:14.435 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.j 2024-05-03T18:13:14.713 INFO:journalctl@ceph.mds.j.smithi122.stdout:May 03 18:13:14 smithi122 systemd[1]: Stopping Ceph mds.j for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:15.213 INFO:journalctl@ceph.mds.j.smithi122.stdout:May 03 18:13:14 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-j[67506]: 2024-05-03T18:13:14.758+0000 7faa0c1a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:15.213 INFO:journalctl@ceph.mds.j.smithi122.stdout:May 03 18:13:14 smithi122 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-j[67506]: 2024-05-03T18:13:14.758+0000 7faa0c1a1700 -1 mds.j *** got signal Terminated *** 2024-05-03T18:13:16.334 INFO:journalctl@ceph.mds.j.smithi122.stdout:May 03 18:13:15 smithi122 podman[89643]: 2024-05-03 18:13:15.98922154 +0000 UTC m=+1.334161620 container died 91076c7dc0123f2c243940a92f6a0aa8cbe78238f20814b1eb399d3d395a3c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-03T18:13:16.334 INFO:journalctl@ceph.mds.j.smithi122.stdout:May 03 18:13:16 smithi122 podman[89643]: 2024-05-03 18:13:16.336163256 +0000 UTC m=+1.681103337 container cleanup 91076c7dc0123f2c243940a92f6a0aa8cbe78238f20814b1eb399d3d395a3c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-j, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240430) 2024-05-03T18:13:16.601 INFO:journalctl@ceph.mds.j.smithi122.stdout:May 03 18:13:16 smithi122 bash[89643]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-j 2024-05-03T18:13:16.759 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.j.service' 2024-05-03T18:13:16.790 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:16.790 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-03T18:13:16.790 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-03T18:13:16.790 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.b 2024-05-03T18:13:17.099 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 18:13:16 smithi149 systemd[1]: Stopping Ceph mds.b for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:17.406 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 18:13:17 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b[63632]: 2024-05-03T18:13:17.100+0000 7f5ea8a41700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:17.407 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 18:13:17 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b[63632]: 2024-05-03T18:13:17.100+0000 7f5ea8a41700 -1 mds.b *** got signal Terminated *** 2024-05-03T18:13:19.290 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 18:13:19 smithi149 podman[69339]: 2024-05-03 18:13:19.016939979 +0000 UTC m=+2.019856440 container died 4c322e9da4d48f97cd8011e836bf9d12b19ca94bd95c7bf9cf15d22227731732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-03T18:13:19.766 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 18:13:19 smithi149 podman[69339]: 2024-05-03 18:13:19.416903519 +0000 UTC m=+2.419819980 container cleanup 4c322e9da4d48f97cd8011e836bf9d12b19ca94bd95c7bf9cf15d22227731732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-03T18:13:19.766 INFO:journalctl@ceph.mds.b.smithi149.stdout:May 03 18:13:19 smithi149 bash[69339]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-b 2024-05-03T18:13:19.892 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.b.service' 2024-05-03T18:13:19.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:19.928 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-03T18:13:19.928 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-03T18:13:19.928 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.e 2024-05-03T18:13:20.040 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 18:13:20 smithi149 systemd[1]: Stopping Ceph mds.e for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:20.541 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 18:13:20 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-e[63001]: 2024-05-03T18:13:20.262+0000 7f0053497700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:20.541 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 18:13:20 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-e[63001]: 2024-05-03T18:13:20.262+0000 7f0053497700 -1 mds.e *** got signal Terminated *** 2024-05-03T18:13:23.819 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 18:13:23 smithi149 podman[69471]: 2024-05-03 18:13:23.549758909 +0000 UTC m=+3.397925678 container died 69442177f39d2420fb56b5ea2bed5d4a16c4ce8478673a0ce5fe0a8506cbd14d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:13:24.158 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 18:13:23 smithi149 podman[69471]: 2024-05-03 18:13:23.91205619 +0000 UTC m=+3.760222966 container cleanup 69442177f39d2420fb56b5ea2bed5d4a16c4ce8478673a0ce5fe0a8506cbd14d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-03T18:13:24.158 INFO:journalctl@ceph.mds.e.smithi149.stdout:May 03 18:13:23 smithi149 bash[69471]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-e 2024-05-03T18:13:24.339 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.e.service' 2024-05-03T18:13:24.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:24.372 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-03T18:13:24.372 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-03T18:13:24.373 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.h 2024-05-03T18:13:24.789 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 18:13:24 smithi149 systemd[1]: Stopping Ceph mds.h for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:25.040 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 18:13:24 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h[63304]: 2024-05-03T18:13:24.791+0000 7f9ae449c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:25.041 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 18:13:24 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h[63304]: 2024-05-03T18:13:24.791+0000 7f9ae449c700 -1 mds.h *** got signal Terminated *** 2024-05-03T18:13:27.281 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 18:13:27 smithi149 podman[69590]: 2024-05-03 18:13:27.024190842 +0000 UTC m=+2.438882853 container died d43d175fadb978164a6e616568282519957830f56de96f023c4b102ff2bef9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-03T18:13:27.541 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 18:13:27 smithi149 podman[69590]: 2024-05-03 18:13:27.382518748 +0000 UTC m=+2.797210829 container cleanup d43d175fadb978164a6e616568282519957830f56de96f023c4b102ff2bef9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-05-03T18:13:27.541 INFO:journalctl@ceph.mds.h.smithi149.stdout:May 03 18:13:27 smithi149 bash[69590]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-h 2024-05-03T18:13:27.771 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.h.service' 2024-05-03T18:13:27.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:27.803 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-03T18:13:27.803 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-03T18:13:27.803 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.k 2024-05-03T18:13:28.040 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 18:13:27 smithi149 systemd[1]: Stopping Ceph mds.k for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:28.540 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 18:13:28 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-k[62694]: 2024-05-03T18:13:28.152+0000 7fab1f8bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:28.541 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 18:13:28 smithi149 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-k[62694]: 2024-05-03T18:13:28.152+0000 7fab1f8bd700 -1 mds.k *** got signal Terminated *** 2024-05-03T18:13:31.951 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 18:13:31 smithi149 podman[69710]: 2024-05-03 18:13:31.677462366 +0000 UTC m=+3.657806900 container died 4a1560d17b8ab054eedd19cd47b3ea148c25cd1a0fe7d54aa37699c2d37dfeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-k, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-03T18:13:32.290 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 18:13:32 smithi149 podman[69710]: 2024-05-03 18:13:32.052145354 +0000 UTC m=+4.032489883 container cleanup 4a1560d17b8ab054eedd19cd47b3ea148c25cd1a0fe7d54aa37699c2d37dfeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-03T18:13:32.291 INFO:journalctl@ceph.mds.k.smithi149.stdout:May 03 18:13:32 smithi149 bash[69710]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-k 2024-05-03T18:13:32.430 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.k.service' 2024-05-03T18:13:32.461 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:32.461 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-03T18:13:32.462 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-03T18:13:32.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.c 2024-05-03T18:13:32.804 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:13:32 smithi192 systemd[1]: Stopping Ceph mds.c for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:33.068 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:13:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c[64201]: 2024-05-03T18:13:32.805+0000 7fc936e69700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:33.068 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:13:32 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c[64201]: 2024-05-03T18:13:32.805+0000 7fc936e69700 -1 mds.c *** got signal Terminated *** 2024-05-03T18:13:35.296 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:13:35 smithi192 podman[70375]: 2024-05-03 18:13:35.028966207 +0000 UTC m=+2.366614472 container died d4bd65c0a1654698980636df1a31bf75d69fed7e37026b630e1541f35cbbf6c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-03T18:13:35.572 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:13:35 smithi192 podman[70375]: 2024-05-03 18:13:35.37238704 +0000 UTC m=+2.710035304 container cleanup d4bd65c0a1654698980636df1a31bf75d69fed7e37026b630e1541f35cbbf6c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-03T18:13:35.572 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:13:35 smithi192 bash[70375]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c 2024-05-03T18:13:35.801 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.c.service' 2024-05-03T18:13:35.822 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:13:35 smithi192 podman[70401]: 2024-05-03 18:13:35.573580301 +0000 UTC m=+0.543853184 container remove d4bd65c0a1654698980636df1a31bf75d69fed7e37026b630e1541f35cbbf6c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:13:35.823 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:13:35 smithi192 systemd[1]: ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.c.service: Deactivated successfully. 2024-05-03T18:13:35.823 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:13:35 smithi192 systemd[1]: Stopped Ceph mds.c for a46fc846-0974-11ef-bc97-c7b262605968. 2024-05-03T18:13:35.823 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 03 18:13:35 smithi192 systemd[1]: ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.c.service: Consumed 2.117s CPU time. 2024-05-03T18:13:35.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:35.835 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-03T18:13:35.836 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-03T18:13:35.836 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.f 2024-05-03T18:13:36.119 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 18:13:35 smithi192 systemd[1]: Stopping Ceph mds.f for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:36.619 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 18:13:36 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f[63882]: 2024-05-03T18:13:36.167+0000 7fb97af93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:36.619 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 18:13:36 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f[63882]: 2024-05-03T18:13:36.167+0000 7fb97af93700 -1 mds.f *** got signal Terminated *** 2024-05-03T18:13:39.369 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 18:13:39 smithi192 podman[70508]: 2024-05-03 18:13:39.025328381 +0000 UTC m=+2.965940221 container died 81729ac8f34527bdcf18b6a5f4d8dba68c2372c7edc32ccdb703b38d66f1b105 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-03T18:13:39.810 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 18:13:39 smithi192 podman[70508]: 2024-05-03 18:13:39.45093391 +0000 UTC m=+3.391545750 container cleanup 81729ac8f34527bdcf18b6a5f4d8dba68c2372c7edc32ccdb703b38d66f1b105 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-03T18:13:39.810 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 18:13:39 smithi192 bash[70508]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f 2024-05-03T18:13:39.810 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 03 18:13:39 smithi192 podman[70521]: 2024-05-03 18:13:39.610482046 +0000 UTC m=+0.579959993 container remove 81729ac8f34527bdcf18b6a5f4d8dba68c2372c7edc32ccdb703b38d66f1b105 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-03T18:13:39.830 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.f.service' 2024-05-03T18:13:39.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:39.861 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-03T18:13:39.862 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-03T18:13:39.862 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.i 2024-05-03T18:13:40.119 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 18:13:39 smithi192 systemd[1]: Stopping Ceph mds.i for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:40.619 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 18:13:40 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i[64511]: 2024-05-03T18:13:40.237+0000 7f649a8d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:40.619 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 18:13:40 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i[64511]: 2024-05-03T18:13:40.237+0000 7f649a8d0700 -1 mds.i *** got signal Terminated *** 2024-05-03T18:13:43.345 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 18:13:43 smithi192 podman[70631]: 2024-05-03 18:13:43.02452313 +0000 UTC m=+2.953899725 container died b336f769e1d5f767ab8c3830994a82cc645a6af16d5c093a2ba7bf388b0fc7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-03T18:13:43.619 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 18:13:43 smithi192 podman[70631]: 2024-05-03 18:13:43.437536849 +0000 UTC m=+3.366913445 container cleanup b336f769e1d5f767ab8c3830994a82cc645a6af16d5c093a2ba7bf388b0fc7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-03T18:13:43.620 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 03 18:13:43 smithi192 bash[70631]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-i 2024-05-03T18:13:43.866 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.i.service' 2024-05-03T18:13:43.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:43.897 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-03T18:13:43.898 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-03T18:13:43.898 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.l 2024-05-03T18:13:44.231 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 18:13:43 smithi192 systemd[1]: Stopping Ceph mds.l for a46fc846-0974-11ef-bc97-c7b262605968... 2024-05-03T18:13:44.621 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 18:13:44 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l[63574]: 2024-05-03T18:13:44.232+0000 7f4d79b03700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:13:44.622 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 18:13:44 smithi192 ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l[63574]: 2024-05-03T18:13:44.232+0000 7f4d79b03700 -1 mds.l *** got signal Terminated *** 2024-05-03T18:13:47.306 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 18:13:47 smithi192 podman[70751]: 2024-05-03 18:13:47.025074113 +0000 UTC m=+2.906408468 container died 67a1d07698113cef427b6253cd7bd8635a13a34286ea0fad7f0b25d719f1876c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:13:47.306 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 18:13:47 smithi192 podman[70751]: 2024-05-03 18:13:47.307765056 +0000 UTC m=+3.189099415 container cleanup 67a1d07698113cef427b6253cd7bd8635a13a34286ea0fad7f0b25d719f1876c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:13:47.619 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 18:13:47 smithi192 bash[70751]: ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l 2024-05-03T18:13:47.620 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 03 18:13:47 smithi192 podman[70765]: 2024-05-03 18:13:47.52530851 +0000 UTC m=+0.497585970 container remove 67a1d07698113cef427b6253cd7bd8635a13a34286ea0fad7f0b25d719f1876c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-a46fc846-0974-11ef-bc97-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-03T18:13:47.656 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a46fc846-0974-11ef-bc97-c7b262605968@mds.l.service' 2024-05-03T18:13:47.691 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:13:47.691 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-03T18:13:47.692 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a46fc846-0974-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T18:13:47.881 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T18:13:50.314 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a46fc846-0974-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T18:13:50.504 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T18:13:52.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a46fc846-0974-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T18:13:53.162 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T18:13:55.450 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:55.482 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:55.513 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:55.538 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-03T18:13:55.541 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410/remote/smithi122/crash 2024-05-03T18:13:55.541 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/crash -- . 2024-05-03T18:13:55.578 INFO:teuthology.orchestra.run.smithi122.stderr:tar: /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T18:13:55.578 INFO:teuthology.orchestra.run.smithi122.stderr:tar: Error is not recoverable: exiting now 2024-05-03T18:13:55.580 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410/remote/smithi149/crash 2024-05-03T18:13:55.581 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/crash -- . 2024-05-03T18:13:55.631 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T18:13:55.631 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-05-03T18:13:55.633 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410/remote/smithi192/crash 2024-05-03T18:13:55.634 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/crash -- . 2024-05-03T18:13:55.664 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/a46fc846-0974-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T18:13:55.664 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2024-05-03T18:13:55.665 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-03T18:13:55.665 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-03T18:13:55.699 INFO:tasks.cephadm:Compressing logs... 2024-05-03T18:13:55.699 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T18:13:55.741 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T18:13:55.744 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T18:13:55.764 INFO:teuthology.orchestra.run.smithi122.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T18:13:55.765 INFO:teuthology.orchestra.run.smithi122.stderr:: No such file or directory 2024-05-03T18:13:55.765 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.log 2024-05-03T18:13:55.770 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T18:13:55.770 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-03T18:13:55.770 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-03T18:13:55.770 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-03T18:13:55.770 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-03T18:13:55.770 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-03T18:13:55.770 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-03T18:13:55.771 INFO:teuthology.orchestra.run.smithi122.stderr: 89.1%/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mgr.x.log: -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T18:13:55.771 INFO:teuthology.orchestra.run.smithi122.stderr: 90.4% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T18:13:55.771 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-03T18:13:55.771 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T18:13:55.786 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.6.log: 94.7% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-03T18:13:55.786 INFO:teuthology.orchestra.run.smithi122.stderr: 94.7% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-03T18:13:55.786 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-03T18:13:55.787 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T18:13:55.787 INFO:teuthology.orchestra.run.smithi122.stderr: 82.9% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T18:13:55.787 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-03T18:13:55.787 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-volume.log: 93.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T18:13:55.787 INFO:teuthology.orchestra.run.smithi122.stderr: 94.6% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-03T18:13:55.787 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-03T18:13:55.787 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.d.log: /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.9.log: 94.7% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-03T18:13:55.789 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T18:13:55.789 INFO:teuthology.orchestra.run.smithi192.stderr:: No such file or directory 2024-05-03T18:13:55.790 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.log 2024-05-03T18:13:55.790 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-03T18:13:55.790 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-03T18:13:55.790 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T18:13:55.790 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-03T18:13:55.790 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.f.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T18:13:55.790 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-03T18:13:55.791 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-03T18:13:55.791 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.5.log: /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-03T18:13:55.791 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-03T18:13:55.791 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mgr.z.log: 89.2% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T18:13:55.791 INFO:teuthology.orchestra.run.smithi192.stderr: 90.6% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T18:13:55.791 INFO:teuthology.orchestra.run.smithi192.stderr: 93.2% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-03T18:13:55.791 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T18:13:55.792 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-03T18:13:55.792 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-03T18:13:55.792 INFO:teuthology.orchestra.run.smithi192.stderr: 82.7% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T18:13:55.792 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T18:13:55.793 INFO:teuthology.orchestra.run.smithi122.stderr: 91.0% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T18:13:55.793 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-03T18:13:55.795 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T18:13:55.795 INFO:teuthology.orchestra.run.smithi149.stderr:: No such file or directory 2024-05-03T18:13:55.796 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.log 2024-05-03T18:13:55.796 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T18:13:55.796 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-03T18:13:55.796 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-03T18:13:55.796 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-03T18:13:55.796 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.7.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T18:13:55.796 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-03T18:13:55.796 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-03T18:13:55.797 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-03T18:13:55.797 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-03T18:13:55.797 INFO:teuthology.orchestra.run.smithi149.stderr: 90.6% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T18:13:55.797 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.h.log: 89.2% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T18:13:55.797 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T18:13:55.797 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-03T18:13:55.797 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.7% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T18:13:55.797 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T18:13:55.797 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.e.log: 93.2% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-03T18:13:55.798 INFO:teuthology.orchestra.run.smithi149.stderr: 94.5% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-03T18:13:55.798 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-03T18:13:55.798 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-03T18:13:55.802 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mon.b.log: /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.k.log: 91.0% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T18:13:55.802 INFO:teuthology.orchestra.run.smithi149.stderr: 94.5% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-03T18:13:55.819 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T18:13:55.825 INFO:teuthology.orchestra.run.smithi192.stderr: 94.0% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-03T18:13:55.829 INFO:teuthology.orchestra.run.smithi192.stderr: 94.1% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-03T18:13:55.855 INFO:teuthology.orchestra.run.smithi149.stderr: 94.6% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-03T18:13:55.856 INFO:teuthology.orchestra.run.smithi149.stderr: 94.5% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-03T18:13:55.887 INFO:teuthology.orchestra.run.smithi192.stderr: 94.6% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-03T18:13:55.930 INFO:teuthology.orchestra.run.smithi192.stderr: 93.9% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-03T18:13:56.114 INFO:teuthology.orchestra.run.smithi122.stderr: 90.3% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-03T18:13:56.216 INFO:teuthology.orchestra.run.smithi192.stderr: 92.2% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-03T18:13:56.216 INFO:teuthology.orchestra.run.smithi149.stderr: 92.1% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-03T18:13:56.452 INFO:teuthology.orchestra.run.smithi122.stderr: 92.5% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-03T18:13:56.712 INFO:teuthology.orchestra.run.smithi149.stderr: 92.2% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-03T18:13:56.750 INFO:teuthology.orchestra.run.smithi192.stderr: 93.4% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-03T18:13:56.797 INFO:teuthology.orchestra.run.smithi122.stderr: 90.3% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-03T18:13:56.812 INFO:teuthology.orchestra.run.smithi149.stderr: 93.3% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-03T18:13:56.859 INFO:teuthology.orchestra.run.smithi192.stderr: 93.2% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-03T18:13:56.871 INFO:teuthology.orchestra.run.smithi122.stderr: 92.9% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-03T18:13:56.887 INFO:teuthology.orchestra.run.smithi149.stderr: 93.0% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-03T18:13:56.978 INFO:teuthology.orchestra.run.smithi192.stderr: 93.1% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-03T18:13:56.988 INFO:teuthology.orchestra.run.smithi149.stderr: 93.1% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-03T18:13:56.991 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-05-03T18:13:56.991 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m1.233s 2024-05-03T18:13:56.991 INFO:teuthology.orchestra.run.smithi149.stderr:user 0m4.808s 2024-05-03T18:13:56.991 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m0.180s 2024-05-03T18:13:57.007 INFO:teuthology.orchestra.run.smithi122.stderr: 93.2% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-03T18:13:57.045 INFO:teuthology.orchestra.run.smithi122.stderr: 93.2% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-03T18:13:57.047 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-05-03T18:13:57.047 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m1.292s 2024-05-03T18:13:57.047 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m5.524s 2024-05-03T18:13:57.048 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.214s 2024-05-03T18:13:57.082 INFO:teuthology.orchestra.run.smithi192.stderr: 92.8% -- replaced with /var/log/ceph/a46fc846-0974-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-03T18:13:57.084 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-05-03T18:13:57.085 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m1.327s 2024-05-03T18:13:57.085 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m5.211s 2024-05-03T18:13:57.085 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m0.176s 2024-05-03T18:13:57.085 INFO:tasks.cephadm:Archiving logs... 2024-05-03T18:13:57.086 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410/remote/smithi122/log 2024-05-03T18:13:57.087 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T18:13:57.976 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410/remote/smithi149/log 2024-05-03T18:13:57.977 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T18:13:58.760 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410/remote/smithi192/log 2024-05-03T18:13:58.761 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T18:13:59.555 INFO:tasks.cephadm:Removing cluster... 2024-05-03T18:13:59.555 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a46fc846-0974-11ef-bc97-c7b262605968 --force 2024-05-03T18:13:59.755 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T18:14:00.012 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a46fc846-0974-11ef-bc97-c7b262605968 --force 2024-05-03T18:14:00.203 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T18:14:00.466 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a46fc846-0974-11ef-bc97-c7b262605968 --force 2024-05-03T18:14:00.655 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: a46fc846-0974-11ef-bc97-c7b262605968 2024-05-03T18:14:00.964 INFO:tasks.cephadm:Removing cephadm ... 2024-05-03T18:14:00.965 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T18:14:00.983 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T18:14:01.000 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T18:14:01.016 INFO:tasks.cephadm:Teardown complete 2024-05-03T18:14:01.016 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-03T18:14:01.026 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-03T18:14:01.027 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T18:14:01.029 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T18:14:01.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T18:14:01.146 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-03T18:14:01.155 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-03T18:14:01.155 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T18:14:01.158 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T18:14:01.161 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T18:14:01.171 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-05-03T18:14:01.174 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-05-03T18:14:01.174 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-03T18:14:01.177 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T18:14:01.177 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-05-03T18:14:01.177 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 +1198us[+1197us] +/- 58ms 2024-05-03T18:14:01.177 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -488us[ -488us] +/- 76ms 2024-05-03T18:14:01.177 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 -1108us[-1109us] +/- 65ms 2024-05-03T18:14:01.178 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:14:01.179 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T18:14:01.180 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-03T18:14:01.180 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 +1237us[+1233us] +/- 58ms 2024-05-03T18:14:01.180 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -450us[ -450us] +/- 76ms 2024-05-03T18:14:01.180 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -1070us[-1070us] +/- 65ms 2024-05-03T18:14:01.180 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:14:01.180 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T18:14:01.180 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-05-03T18:14:01.181 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv01.front.sepia.ceph.com 3 6 377 17 +1199us[+1201us] +/- 58ms 2024-05-03T18:14:01.181 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -483us[ -481us] +/- 76ms 2024-05-03T18:14:01.181 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -1104us[-1104us] +/- 65ms 2024-05-03T18:14:01.181 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:14:01.181 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-03T18:14:01.191 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-03T18:14:01.191 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-03T18:14:01.201 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T18:14:01.293 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T18:14:01.363 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T18:14:01.427 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T18:14:01.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:14:01.455 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T18:14:01.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:14:01.486 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T18:14:01.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:14:01.512 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-03T18:14:01.522 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-03T18:14:01.536 INFO:teuthology.task.internal:Duration was 2211.128039 seconds 2024-05-03T18:14:01.536 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-03T18:14:01.546 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-03T18:14:01.546 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T18:14:01.548 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T18:14:01.551 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T18:14:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T18:14:01.587 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T18:14:01.588 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T18:14:02.088 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-03T18:14:02.088 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-05-03T18:14:02.089 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T18:14:02.109 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-05-03T18:14:02.109 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T18:14:02.155 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-05-03T18:14:02.155 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T18:14:02.188 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-03T18:14:02.188 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T18:14:02.191 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T18:14:02.198 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T18:14:02.253 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-03T18:14:02.253 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T18:14:02.258 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T18:14:02.262 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T18:14:02.761 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-03T18:14:02.772 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-03T18:14:02.772 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T18:14:02.796 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T18:14:02.820 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T18:14:02.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-03T18:14:02.855 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T18:14:02.857 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T18:14:02.863 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T18:14:02.876 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-05-03T18:14:02.883 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-05-03T18:14:02.906 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-05-03T18:14:02.945 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T18:14:02.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:14:02.959 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T18:14:02.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:14:02.973 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T18:14:02.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:14:02.999 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-03T18:14:03.009 INFO:teuthology.task.internal:Transferring archived files... 2024-05-03T18:14:03.010 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410/remote/smithi122 2024-05-03T18:14:03.010 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T18:14:03.058 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410/remote/smithi149 2024-05-03T18:14:03.058 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T18:14:03.092 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410/remote/smithi192 2024-05-03T18:14:03.092 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T18:14:03.127 INFO:teuthology.task.internal:Removing archive directory... 2024-05-03T18:14:03.127 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T18:14:03.130 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T18:14:03.132 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T18:14:03.175 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-03T18:14:03.187 INFO:teuthology.task.internal:Not uploading archives. 2024-05-03T18:14:03.187 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-03T18:14:03.198 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-03T18:14:03.198 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T18:14:03.201 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T18:14:03.203 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T18:14:03.213 INFO:teuthology.orchestra.run.smithi122.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 18:14 /home/ubuntu/cephtest 2024-05-03T18:14:03.215 INFO:teuthology.orchestra.run.smithi149.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 18:14 /home/ubuntu/cephtest 2024-05-03T18:14:03.229 INFO:teuthology.orchestra.run.smithi192.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 18:14 /home/ubuntu/cephtest 2024-05-03T18:14:03.231 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-03T18:14:03.242 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-03T18:14:03.263 INFO:teuthology.nuke:Checking targets against current locks 2024-05-03T18:14:03.290 DEBUG:teuthology.nuke:shortname: smithi122 2024-05-03T18:14:03.290 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T18:14:03.314 DEBUG:teuthology.nuke:shortname: smithi149 2024-05-03T18:14:03.314 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T18:14:03.319 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 17:31:24.231015', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGs35vtwL+0yN/J032ku2Vt6SdDjMgN8+RiuzTk3cZUfyPOCWi8tOZ5LJUUmfidtytXDavqgYJU1bHin+WcvHfo='} 2024-05-03T18:14:03.335 DEBUG:teuthology.nuke:shortname: smithi192 2024-05-03T18:14:03.335 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T18:14:03.342 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 17:31:24.227649', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOH20P6Jxshe2lqJN0Pt/+RuGGcCp2X+1Fns/lvH7xoS6GQFxJP8Scvtex/LQ/7+BiwnIpZuoNUHPz4VCPgrWzo='} 2024-05-03T18:14:03.354 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687410', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 17:31:24.229542', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDTAhAyIIG5W3tltDH9m3ykiOIeWdXDXdlSrM1UbDQNYe1xtmP+UND6uYbxt3buulING/ZqQTsAFAsJP4FkjwJU='} 2024-05-03T18:14:03.364 INFO:teuthology.orchestra.console.smithi122:Power off 2024-05-03T18:14:03.365 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T18:14:03.393 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2024-05-03T18:14:03.393 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T18:14:03.417 INFO:teuthology.orchestra.console.smithi192:Power off 2024-05-03T18:14:03.417 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T18:14:03.435 INFO:teuthology.orchestra.console.smithi149:Power off 2024-05-03T18:14:03.436 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T18:14:03.446 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-05-03T18:14:03.446 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2024-05-03T18:14:03.446 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T18:14:03.461 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2024-05-03T18:14:03.462 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T18:14:03.471 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-05-03T18:14:03.485 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-05-03T18:14:07.447 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T18:14:07.472 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T18:14:07.486 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T18:14:07.560 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-05-03T18:14:07.581 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-05-03T18:14:07.596 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-05-03T18:14:11.565 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T18:14:11.582 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T18:14:11.597 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T18:14:11.676 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-05-03T18:14:11.692 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2024-05-03T18:14:11.709 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-05-03T18:14:11.793 INFO:teuthology.orchestra.console.smithi192:Power off completed 2024-05-03T18:14:15.679 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T18:14:15.709 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T18:14:15.790 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-05-03T18:14:15.823 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-05-03T18:14:15.891 INFO:teuthology.orchestra.console.smithi122:Power off completed 2024-05-03T18:14:15.924 INFO:teuthology.orchestra.console.smithi149:Power off completed 2024-05-03T18:14:16.030 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} duration: 2211.1280393600464 failure_reason: 'Command failed on smithi122 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c ''ceph fs subvolumegroup pin cephfs qa random 0.10''"' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=fb7b858d864d4e5cb209641204f79fe7 status: fail success: false 2024-05-03T18:14:16.030 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-03T18:14:16.112 INFO:teuthology.run:FAIL